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Abstract 


This  report  documents  the  completion  of  Task  3  of  the  work  stream  “Research  Using  in  Vivo 
Simulation  of  Meta-Organizational  Shared  Decision  Making  (SDM)”,  one  component  of  the 
Technology  Innovation  Fund  (TIF)  program  on  Meta-organizational  Collaboration  that  has  been 
designed  to  assist  in  understanding  challenges  faced  by  the  Canadian  Forces  (CF).  The  objective 
of  the  stream  is  to  conduct  basic  research  into  shared  decision  making  through  the  analysis  of 
case  studies,  exercises  and  simulations.  Task  3  involved  the  development  and  testing  of  the 
shared  decision  making  framework  in  vivo.  The  research  at  this  stage  is  to  demonstrate  that  the 
model  when  implemented  in  vivo  can  produce  improvements  in  problem  solving  processes  and 
outcomes  such  as  better  quality  decisions,  higher  levels  of  satisfaction  with  problem  solving 
processes,  better  time-to-satisfaction  ratio  and  more  cohesive  multi- organization  groups. 


Resume 


Ce  rapport  traite  de  F execution  de  la  Tache  3  du  projet  intitule  :  «  Recherche  par  la  simulation  in 
vivo  sur  la  prise  de  decision  partagee  des  meta-organisations  »,  une  des  composantes  du 
programme  du  Fonds  pour  F  innovation  technologique  (FIT)  relatif  a  la  collaboration  meta- 
organisationnelle,  qui  a  ete  congu  afm  d’ameliorer  la  comprehension  des  defis  auxquels  font  face 
les  Forces  canadiennes  (FC).  L’objectif  de  ce  volet  particulier  est  de  mener  une  recherche  de  base 
sur  le  partage  des  decisions  au  moyen  d’etudes  de  cas,  d’exercices  et  de  simulations.  La  Tache  3 
consistait  a  e  laborer  le  cadre  de  partage  des  decisions  in  vivo  et  d’en  faire  l’essai.  A  ce  stade,  la 
recherche  visait  a  demontrer  que  le  modele,  lorsqu’il  est  mis  en  oeuvre  in  vivo,  peut  aider  a 
ameliorer  les  processus  de  resolution  des  problemes  et  leurs  resultats,  notamment  des  decisions  de 
meilleure  qualite,  de  hauts  niveaux  de  satisfaction  en  ce  qui  touche  les  processus  de  resolution  des 
problemes,  un  meilleur  rapport  temps-satisfaction  et  des  groupes  multi-organisationnels  plus 
cohesifs. 
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Executive  summary 


Research  Using  In  Vivo  Simulation  of  Meta-Organizational 
Shared  Decision  Making  (SDM)  -  Task  3:  Testing  the  Shared 
Decision  Making  Framework  in  vivo 

Louise  Lemyre1  et  al.;  DRDC  CSS  CR  2011-32;  Defence  R&D  Canada  -  CSS. 

Introduction:  This  report  documents  the  completion  of  Task  3  of  the  work  stream  “Research 
Using  in  Vivo  Simulation  of  Meta-Organizational  Shared  Decision  Making  (SDM)”,  one 
component  of  the  Technology  Innovation  Fund  (TIF)  program  on  Meta-organizational 
Collaboration  that  has  been  designed  to  assist  in  understanding  challenges  faced  by  the  Canadian 
Forces  (CF).  The  objective  of  the  stream  is  to  conduct  basic  research  into  shared  decision  making 
through  the  analysis  of  case  studies,  exercises  and  simulations. 

The  objective  of  Task  3  is  to  test  the  shared  decision  making  framework  in  vivo. 

Method:  Headed  by  Dr.  Lemyre,  the  Gap-Sante  research  team  at  the  University  of  Ottawa 
created  a  Model  for  Inter-organizational  Problem  Solving  under  Task  1  of  the  project.  Task  2 
involved  the  development  of  a  research  plan  for  an  in  vivo  simulation  experiment  as  well  as 
qualitative  interviews.  Task  3  comprised  testing  the  shared  decision-making  (SDM)  framework 
using  qualitative  interviews  with  key  decision  makers  having  played  a  role  in  major  events  and 
testing  an  in  vivo  experimental  simulation  of  shared  decision  making  in  a  complex  scenario. 

Results:  The  research  at  this  stage  is  to  demonstrate  that  the  model  when  implemented  in  vivo 
can  produce  improvements  in  problem  solving  processes  and  outcomes  such  as  better  quality 
decisions,  higher  levels  of  satisfaction  with  problem  solving  processes,  better  time-to-satisfaction 
ratio  and  more  cohesive  multi-organization  groups. 


1  Dr.  Lemyre  is  the  McLaughlin  Research  Chair  on  Psychosocial  Aspects  of  Risk  and  Health,  Director  of 
GAP-Sante  at  the  Institute  of  Population  Health,  and  Professor  of  Psychology,  Faculty  of  Social  Science, 
University  of  Ottawa 
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Sommaire 


Recherche  sur  le  partage  de  decision  des  meta-organisations  en 
utilisant  la  simulation  in  vivo  -  Tache  3  :  Essai  du  cadre  de 
partage  des  decisions  in  vivo 

Louise  Lemyre2  et  autres  ;  CSS  RDDC  CR  2011-32  ;  R&D  pour  la  defense 
Canada  -  CSS. 

Introduction:  Ce  rapport  traite  de  l’execution  de  la  Tache  3  du  projet  intitule  :  «  Recherche  par 
la  simulation  in  vivo  sur  la  prise  de  decision  partagee  des  meta-organisations  »,  une  des 
composantes  du  programme  du  Fonds  pour  Tinnovation  technologique  (FIT)  relatif  a  la 
collaboration  meta-organisationnelle,  qui  a  ete  congu  afm  d’ameliorer  la  comprehension  des  defis 
auxquels  font  face  les  Forces  canadiennes  (FC).  L’objectif  de  ce  volet  particulier  est  de  mener 
une  recherche  de  base  sur  le  partage  des  decisions  au  moyen  d’etudes  de  cas,  d’exercices  et 
de  simulations. 

La  Tache  3  a  pour  objectif  de  faire  l’essai  du  cadre  de  partage  des  decisions  in  vivo. 

Methode  :  Sous  la  direction  de  Mme  Lemyre,  Ph.  D.,  l’equipe  de  recherche  GAP-Sante  de 
l’Universite  d’Ottawa  a  cree  un  modele  de  resolution  inter-organisationnelle  de  problemes  dans 
le  cadre  de  la  Tache  1  du  projet.  La  Tache  2  consistait  a  e  laborer  un  plan  de  recherche  pour  une 
experience  de  simulation  in  vivo  ainsi  que  des  entrevues  qualitatives.  La  Tache  3,  quant  a  elle, 
portait  sur  l’essai  du  cadre  de  partage  des  decisions  au  moyen  d’entrevues  qualitatives  realisees 
aupres  des  principaux  decideurs  ayant  joue  un  role  dans  des  activites  d’importance  et  sur  l’essai 
d’une  simulation  experimentale  in  vivo  du  partage  des  decisions  dans  un  scenario  complexe. 

Resultats  :  A  ce  stade,  ;a  recherche  visait  a  demontrer  que  le  modele,  lorsqu’il  est  mis  en  oeuvre 
in  vivo,  peut  ameliorer  les  processus  de  resolution  de  problemes  et  leurs  resultats,  notamment  des 
decisions  de  meilleure  qualite,  de  hauts  niveaux  de  satisfaction  en  ce  qui  touche  les  processus  de 
resolution  des  problemes,  un  meilleur  rapport  temps-satisfaction  et  des  groupes  multi- 
organisationnels  plus  cohesifs. 


2  Mme  Lemyre,  Ph.  D.,  est  titulaire  de  la  Chaire  de  recherche  McLaughlin  sur  le  risque  psychosocial, 
directrice  de  GAP-Sante,  a  l’lnstitut  de  sante  des  populations  et  professeure  de  psychologie  a  la  faculte  des 
sciences  sociales  de  FUniversite  d’Ottawa. 
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Preface 


This  document  represents  the  initial  draft  of  Task  3:  Testing  the  Shared  Decision  Making 
Framework  in  vivo,  and  is  submitted  for  review  before  a  final  document  is  prepared  using  the 
DRDC  supplied  template  for  Contractor  reports.  The  work  has  been  completed  for  Defence 
Research  and  Development  Canada  (DRDC)  as  part  of  the  contract  deliverable  defined  in  the 
project  entitled  Research  Using  In-Vivo  Simulation  of  Meta-Organizational  Shared  Decision 
Making  (SDM),  Contract  No.:  W7714-083659/001/SV. 

The  document  is  a  technical  report  that  provides  detailed  descriptions  of  all  of  the  materials, 
instruments,  procedures  and  processes  created  to  implement  both  the  qualitative  interview 
component  of  the  study  and  the  in  vivo  simulation  exercise.  The  results  of  the  scenario  assessment 
process  for  determining  complexity  are  included.  Also  included  in  this  report  are  results  from  the 
multiple  pilot  testing  of  the  simulation  using  different  configurations  of  participants  and  pods  and 
the  plans  formulated  for  conducting  simulation  exercise  sessions  with  senior  decision  makers  in 
early  April  and  May  2011.  A  follow-on  report  is  planned  and  will  include  detailed  analyses  and 
results  for  both  the  qualitative  interview  component  and  the  senior  level  sessions  of  the  in  vivo 
experiment. 

Document  Distribution  and  Confidentiality 

Document  distribution  and  confidentiality  protocols  as  specified  in  the  contract  noted  above  will 
apply  to  this  document.  Please  contact  Dr.  Louise  Lemyre,  Principal  Investigator,  University  of 
Ottawa,  at  louise.lemyre@uOttawa.ca  should  a  change  in  protocols  be  requested.  Please  quote 
with  due  reference  to  Lemyre  et  al.  2011,  Report  on  Research  Using  in  Vivo  Simulation  of  Meta- 
Organizational  Shared  Decision  Making  (SDM)  -  Task  3:  Testing  the  Shared  Decision  Making 
Framework  in  vivo. 
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1  Introduction 


This  draft  report  documents  the  completion  status  and  results  to-date  of  Task  3:  Testing  the 
shared  decision  making  framework  in  vivo.  Task  3  is  one  of  five  tasks  to  be  undertaken  to 
conduct  basic  research  into  shared  decision  making  through  the  analysis  of  case  studies,  exercises 
and  simulations.  It  builds  on  work  completed  under  Task  1:  Synthesis  of  Case  Studies  to  form  a 
SDM  Framework  (see  figure  below)  and  reflects  the  outputs  of  Task  2:  Development  of  an 
experimental  plan  for  in  vivo  exercise  and  simulation.  Refer  to  report  entitled.  Research  Using  In 
Vivo  Simulation  of  Meta-organizational  Shared  Decision  Making  (SDM)  Task  1:  Synthesis  of 
Case  Studies  to  form  a  SDM  Framework,  (Lemyre  et  al.,  2009)  for  details  on  Task  1. 


Situation 

Intarorganteationai 

Approach 

Figure  1:  Model  of  inter-organizational  problem  solving 

This  report  focuses  on  the  implementation  procedures  and  protocols  for  both  the  qualitative 
interview  component  of  Task  3  and  the  in  vivo  simulation  experiment.  Detailed  descriptions  are 
provided  of  all  materials  utilised  during  the  course  of  the  testing  undertaken  to-date  and  the 
interview  processes.  While  all  materials  and  processes  have  been  validated  through  extensive 
rounds  of  pilot  testing,  experimental  sessions  with  senior  decision-makers  are  presently  being 
planned  for  April  and  May  2011.  Given  the  required  timing  of  this  report,  results  for  the 
experimental  component  will  be  limited  in  this  report  to  relevant  themes  and  observations  from 
the  multiple  pilot  tests  that  have  been  undertaken  during  Task  3.  All  of  the  qualitative  interviews 
have  been  conducted  but  these  are  in  the  transcription  and  coding  processes.  After  completion  of 
these  activities,  analyses  will  be  undertaken  and  findings  integrated  with  those  from  the  in  vivo 
experimental  sessions  with  senior  officials. 

1.1  Task  3  objective  and  components 

The  objective  of  Task  3  is  to  test  the  shared  decision  making  framework  in  vivo.  From  the  results 
of  Task  1,  identify  scenarios  that  challenge  the  SDM  framework,  as  well  as  the  ICS  framework,  if 
the  latter  differs  from  the  SDM  framework.  These  scenarios  will  be  exercised  within  the 
simulated  environment  designed  in  Task  2.  Participants  in  the  simulations  will  include  individuals 
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from  multiple  levels  of  government,  multiple  jurisdictions  and  multiple  disciplines  according  to 
the  requirements  of  the  particular  scenario  being  exercised. 


1.2  Research  objectives,  questions  and  strategy 

The  research  objectives,  questions  and  strategy  were  developed  as  part  of  Task  2  and  a  full 
discussion  can  be  found  in  the  report  entitled  Research  Using  In  Vivo  Simulation  of  Meta- 
organizational  Shared  Decision  Making  (SDM)  Task  2:  Development  of  an  experimental  plan  for 
in  vivo  exercise  and  simulation  (Lemvre  et  ah,  2010).  The  figure  below  summarizes  this 
information  and  is  re-presented  for  reader  convenience. 

The  research  at  this  stage  is  to  demonstrate  that  the  model  when  implemented  in  vivo  can  produce 
improvements  in  problem  solving  processes  and  outcomes  such  as  better  quality  decisions,  higher 
levels  of  satisfaction  with  problem  solving  processes,  better  time-to-satisfaction  ratio  and  more 
cohesive  multi-organization  groups. 
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Figure  2:  Overview  of  research  strategy > 


1 .3  Overview  of  report  sections 

This  document  is  organized  into  three  main  sections  with  various  sub-sections.  The  content  for 
each  of  these  sections  is  briefly  described  below. 


SECTION  1  Introduction 

•  This  section  outlines  the  objectives  of  Task  3  and  its  qualitative  and  experimental 
components. 

•  Details  are  provided  concerning  the  study's  research  objectives,  questions  and  strategies. 
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•  The  three  sections  of  the  report  and  what  each  section  will  contain  are  described. 


SECTION  2  Component  1  -  Qualitative  interviews  with  decision  makers 

•  This  section  outlines  the  qualitative  interview  research  design  and  its  rationale. 

•  Details  are  provided  concerning  the  ethics  approval  process,  the  interview  guide 
development  and  its  contents,  and  the  interviewee  recruitment  procedure. 

•  The  data  collection  and  analysis  process  is  described. 

•  Next  steps  for  the  qualitative  interviews  focus  on  the  directions  of  the  final  analyses. 


SECTION  3  Component  2  -  In  Vivo  Simulation  experiment 

•  This  section  outlines  the  in  vivo  simulation  experimental  research  design,  and  briefly 
describes  the  control,  independent,  and  dependent  variables. 

•  Details  are  provided  concerning  ethics  approval,  the  ethics  approval  process  as  well  as 
proper  data  storage  and  maintaining  participant  confidentiality  through  the  use  of  participant 
identifiers. 

•  The  rational  for  rating  scenario  complexity  and  its  relation  to  the  shared  decision  making 
model  are  discussed.  The  scenario  development  process,  the  scenario  assessment  and  its 
results,  as  well  as  the  scenario  selection  and  refinement  process  are  presented.  Refinement 
of  the  rating  scale  for  future  research  purposes  is  discussed. 

•  Developed  instruments  such  as  dependent  measures,  the  background  questionnaire,  and  the 
decision  quality  rating  instrument  are  presented  and  described  in  terms  of  their  development 
process,  design  considerations,  review  process,  and  their  refinement. 

•  Development  and  assessment  of  simulation  tasks  are  discussed.  Details  are  presented  on  the 
rational  for  assessing  simulation  tasks,  simulation  tasks  relation  to  the  shared  decision 
making  model,  and  the  task  development  and  assessment  process.  Worksheets  that 
correspond  to  the  chosen  tasks  are  presented. 

•  Details  are  provided  concerning  the  development  of  the  multimedia  simulations,  POD 
session  roles,  responsibilities  and  materials,  recruitment  methods,  necessary  equipment, 
facilities  and  software,  and  data  collection  and  storage  procedures. 

•  A  pilot  testing  overview  is  presented  along  with  results  and  information  regarding  changes 
made  to  simulation  materials. 


SECTION  4  Next  Steps _ 

•  This  section  provides  an  overview  of  the  remaining  scheduled  sessions  with  senior 
emergency  management  professionals. 

•  Data  collection  methods  and  planned  analysis  are  also  described  with  respect  to  both  the 
qualitative  interview  component  and  the  in  vivo  simulation  component  of  the  project. 

•  Finally,  the  upcoming  task  of  modelling  communication  and  decision  making  functions  is 
described. 
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2  Component  1  -  Qualitative  interviews  with  decision 
makers 


This  section  of  the  report  presents  an  overview  of  the  qualitative  interview  component  of  the 
project.  The  following  items  are  described  here:  overview  of  the  research  design,  ethics  approval, 
interview  guide  development,  recruitment,  data  collection,  data  analysis,  and  next  steps. 

2.1  Overview  of  research  design 

Qualitative  research  methods  are  often  complementary  to  using  quantitative  methods.  Qualitative 
methods  elicit  explanations,  discover  patterns,  and  understand  relationships  in  the  data  that  may 
be  overlooked  when  only  using  quantitative  methods.  Qualitative  methods  help  go  beyond  the 
scope  of  quantitative  methods  (Crotty,  1998;  Smith,  2008).  The  present  study  uses  qualitative 
findings  from  interviews  with  key  stakeholders  as  a  method  to  identify  critical  insights  into 
elements  of  decision  making  processes  during  extreme  events  in  real-life  settings  retrospectively 
(Nja  &  Rake,  2009)  to  aid  in  the  planning,  validation,  analysis  and  interpretation  of  the 
quantitative  in  vivo  experimental  data.  The  interviews  in  the  present  study  focus  on  the  general 
process  of  problem  solving,  during  which  decision  making  is  considered  to  be  one  of  the  key 
stages. 

The  present  study  uses  the  Critical  Decision  Method  approach  to  interviewing  (Klein, 
Calderwood  &  MacGregor,  1989)  which  has  been  simplified  and  adapted  by  other  authors  (e.g, 
Smith  and  Dowell,  2000;  Nja  &  Rake,  2009).  A  semi-structured  interview  guide  consisting  of 
core  questions  and  probes  (Lindlof  &  Taylor,  2002)  was  used  to  conduct  the  interviews.  The 
interviewer  guided  interviewees  through  the  various  stages  of  problem  solving  that  had  occurred 
during  a  previous  extreme  event,  drawing  out  the  interviewees’  past  experiences  and 
interpretations.  The  main  strengths  and  challenges  to  using  this  approach  were  discussed  in  the 
previous  report  “Research  Using  In  Vivo  Simulation  of  Meta- Organizational  Shared  Decision 
Making  (SDM):  Task  2:  Development  of  an  experimental  plan  for  in  vivo  exercise  and 
simulation”. 


2.2  Ethics  approval 

The  qualitative  interview  component  of  the  study  obtained  ethics  approval  from  the  Social 
Sciences  and  Humanities  Research  Ethics  Board  of  the  University  of  Ottawa.  For  details  on  the 
application  and  ethics  submission  please  refer  to  Section  3.2. 

2.3  Interview  guide  development 

A  semi- structured  interview  guide  allows  new  questions  to  be  asked  based  on  the  interviewees’ 
conversational  direction  (Lindlof  &  Taylor,  2002).  For  the  present  study,  a  guide  was  developed 
to  instruct  the  administration  and  implementation  of  the  qualitative  interviews.  The  guide  was 
used  for  all  interviews  to  ensure  some  consistency  across  interviews  and  increase  the  reliability  of 
findings.  The  draft  guide  went  through  various  iterations  and  pre-testing  to  ensure  that  the 
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interview  was  of  an  appropriate  length  (approximately  60  minutes),  and  that  questions  collected 
appropriate  information  from  interviewees,  were  easy  to  understand,  and  flowed  in  a  logical 
fashion. 

The  questions  in  the  interview  guide  were  based  on  the  current  study's  two  main  research 
questions  described  in  Section  1.2.  Nine  interview  questions  along  with  additional  probes  were 
developed  to  address  the  two  main  research  questions  (see  Annex  A).  Exact  wording  of  the 
questions  varied  during  interviews  to  accommodate  the  different  situations  or  events  being 
studied.  The  interview  began  with  a  brief  introduction  to  convey  some  context  to  the 
interviewees,  and  was  followed  by  some  questions  designed  to  collect  some  basic,  factual 
information  concerning  the  interviewees’  background,  an  overview  description  of  the  selected 
event,  and  the  selection  of  one  or  two  challenging  situations  that  had  complex  and  multi- 
organizational  characteristics.  Once  one  or  two  situations  had  been  selected,  the  interview 
focused  on  the  chosen  situations  with  questions  designed  to  have  the  interviewee  describe  and 
reflect  upon  problem  identification  and  definition,  solution  generation,  decision  making  and 
implementation,  sharing  resources  and  flexibility,  expectations  and  alternatives,  and  the  three 
concepts  of  coordination,  cooperation  and  collaboration. 

2.4  Recruitment  of  interviewees 

The  inclusion  criteria  for  participants  for  the  qualitative  interviews  were  that  they  were  senior- 
level  decision  makers  and  managers  who  had  participated  in  planning  for  and/or  managing  a 
major,  extreme  event  in  Canada  within  the  past  ten  years.  They  would  need  to  have  played  a 
strategic  decision-making  role  during  at  least  one  major  event  such  as  the  2010  G-20  Toronto 
Summit,  2010  Vancouver  Olympic  Games,  the  2009  H1N1  planning,  the  2007  G8  meeting  in 
Ottawa,  or  the,  SARS  outbreak  in  2003.  Other  considerations  included  level  of  experience 
managing  extreme  events,  and  possessing  the  authority  to  make  decisions  regarding  allocation  of 
resources  on  behalf  of  their  organization  with  respect  to  major  events.  Some  potential 
interviewees  were  identified  via  the  professional  networks  of  research  team  members.  Others 
were  identified  through  referrals  from  key  senior  managers  and  decision-makers  who  participated 
in  the  same  events. 

One  goal  of  the  interviews  was  to  ensure  that  there  was  some  diversity  among  interviewees  with 
respect  to  the  type  of  decision/command  structures  that  were  characteristic  of  their  organizations. 
As  a  result,  an  effort  was  made  to  recruit  key  decision  makers  from  a  variety  of  organizations  that 
follow  different  command  structures.  Three  types  of  organizations  were  targeted:  1)  military 
organizations  that  employ  a  C2  structure  (e.g.  Canadian  Forces);  2)  service  organizations  that  rely 
primarily  on  the  Incident  Command  System  (ICS)  approach  to  planning  for  and  responding  to 
extreme  events  (e.g.,  EMO);  and  3)  organizations  that  follow  less  hierarchical,  more  distributed 
and  less  prescriptive  decision-making  processes  (e.g.,  NGOs,  community  organizations,  health 
authorities). 

Given  the  inherent  challenges  involved  in  the  recruitment  of  senior-level  decision  makers,  a 
flexible  recruitment  process  was  developed  involving  various  stages  and  procedures  which 
included  identifying  potential  interviewees,  making  initial  contact  (see  Annex  B  for  the  e-mail 
invitation),  securing  agreement  to  participate,  scheduling  interviews,  obtaining  informed  consent, 
and  conducting  the  interviews. 
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2.5  Data  collection 


All  interviews  were  conducted  by  senior-level  interviewers  with  advanced  knowledge  and 
experience  with  qualitative  interviewing  techniques.  Prior  to  the  interview  starting,  interviewees 
were  provided  with  detailed  information  about  the  study  via  an  information  sheet  that 
accompanied  the  consent  form.  The  information  and  consent  form  was  approved  by  the 
University  of  Ottawa  Research  Ethics  Board  and  is  in  compliance  with  the  Tri-Council  Policy 
Statement:  Ethical  Conduct  for  Research  Involving  Humans  (TCPS).  The  interviewer  would 
guide  the  interviewee  through  the  informed  consent  process,  asking  them  if  they  had  any 
questions  or  required  any  further  clarification  prior  to  signing  the  form  indicating  their  consent  to 
participate  in  the  interview.  (See  Annex  C  for  the  participant  consent  form). 

Once  informed  consent  was  obtained,  the  interviewer  proceeded  with  the  interview  using  the 
interview  guide.  (See  Annex  A  for  the  qualitative  interview  guide).  Interviews  were  semi- 
structured,  with  open-ended  questions  asked,  followed  by  probes  designed  to  elicit  more  detailed 
information.  All  interviews  were  audio-taped,  with  tapes  later  transcribed  verbatim. 

As  of  the  timing  of  this  report,  ten  participants  had  been  recruited  and  interviewed,  eight  in  the 
Toronto  area  and  two  in  the  National  Capital  Region,  from  military,  ICS  and  non-ICS 
organizations.  Interviews  lasted  on  average  60  minutes  in  duration. 

2.6  Data  analysis 

In  qualitative  research,  often  a  larger  emphasis  is  placed  on  the  depth  and  details  of  the  data  than 
on  the  breadth  and  representativeness  of  data.  Each  qualitative  interview  is  treated  as  a  rich 
source  of  information  that  broadens  the  researcher’s  understanding,  that  when  combined  with  the 
information  collected  from  other  interviews  will  demonstrate  patterns  and  themes,  along  with 
explanations  and  examples. 

The  verbatim  transcripts  of  the  interviews  were  imported  into  an  analytic  software  program 
designed  to  assist  with  categorizing,  and  assembling  qualitative  information  so  that  trends  and 
patterns  can  be  analyzed.  Initially,  analysis  of  the  transcripts  began  with  identifying  meaningful 
units  of  information  that  are  related  to  situation  complexity  (i.e.,  simple,  complicated,  and 
complex).  This  was  followed  by  identifying  units  of  information  by  expected  themes  (e.g.,  multi 
organizational  environment,  approach  to  problem  solving,  problem  solving  stage)  and  emerging 
themes.  Data  coding  (i.e.,  assigning  text  or  units  of  information  to  categories)  was  conducted  in  a 
cascading  manner,  where  data  was  categorized  into  a  small,  broad  number  of  categories. 

2.7  Next  steps 

At  the  timing  of  this  report,  recorded  interviews  are  currently  being  transcribed.  Once  the 
transcriptions  have  been  completed,  the  detailed  analyses  will  begin.  All  transcripts  will  be 
imported  into  analytic  software  and  follow  the  analysis  process  as  described  in  Section  2.6  Data 
Analysis.  Once  a  small  number  of  broad  categories  are  identified  from  the  interviews,  these 
categories  will  be  coded  into  additional  levels  of  subcategories  to  specify  certain  themes.  To 
ensure  reliability  of  the  coding  scheme  and  analysis,  two  researchers  will  recode  10%  of  the 
interviews,  and  then  these  coding  results  will  be  compared.  Finally,  results  will  be  linked  with  the 
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quantitative  experimental  in  vivo  component,  and  will  aid  in  the  interpretation  of  the  quantitative 
findings. 
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3  Component  2  -  In  vivo  simulation  experiment 


This  section  describes  the  second  major  component  of  the  project  -  the  in  vivo  simulation 
experiment.  An  overview  of  the  research  design  for  the  in  vivo  experiment  is  presented  along 
with  a  description  of  the  ethics  approval  process.  This  portion  of  the  report  also  describes  our 
methods  for  determining  situational  complexity;  the  instruments  used  in  the  experiment;  scenario 
and  task  development;  the  development  of  the  final  multimedia  version  of  the  simulation 
scenario;  the  roles,  responsibilities  and  materials  associated  with  each  of  the  actors  involved  in 
the  experiment;  the  recruitment  process;  the  equipment,  facilities,  software  and  set-up  required  to 
run  the  experiment;  as  well  as  a  description  of  the  pilot  testing  phases  of  the  experiment. 

3.1  Overview  of  research  design 

In  order  to  test  the  model  for  inter-organizational  problem  solving,  an  in  vivo  simulation 
experiment  was  developed.  This  simulation  experiment  is  in  keeping  with  Jonathon  Crego’s  work 
on  the  HYDRA  simulation  training  system  (Alison  &  Crego,  2008);  however,  the  Inter-GAP  In 
Vivo  System  developed  by  Lemyre  et  al.  (2010)  looks  not  only  at  zntra-organizational  problem 
solving  but  also  at  zwter-organizational  problem  solving.  The  research  design  assumes  a  complex 
situation  within  which  decision  makers  have  to  undertake  a  number  of  tasks  characterized  as 
either  “coordination”  or  “collaboration”. 

During  an  Inter-GAP  simulation  system  session,  groups  of  three  participants  are  organized  into 
pods  which  are  equipped  with  communication  equipment.  These  teams  work  through  a  simulated 
emergency  event  while  their  communications,  interactions  (both  within  pods  and  between  pods) 
as  well  as  the  decision  making  processes  are  recorded  for  later  analysis.  Three  of  the  teams  are 
able  to  communicate  between  the  pods,  while  a  fourth  team  is  used  as  the  control  group,  and  is 
not  given  the  communication  tools  to  interact  with  the  other  pods  (see  Figure  3  below  for  the 
overall  session  composition).  The  session  is  delivered  via  a  separate  control  room  using  video 
conferencing  software,  where  the  scenario,  tasks,  and  injects  are  delivered  according  to  a  set 
schedule.  Facilitator-led  briefing  and  debriefing  sessions  are  used  to  orient  the  participants  to  the 
materials  and  technology  used  in  the  experiment  and  to  gain  informed  consent  from  each 
participant. 
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Figure  3:  Overall  session  composition 


3.1.1  Description  of  variables 

The  independent  variables  that  were  manipulated  during  the  experiment  are  a)  Approach  to 
problem  solving,  and  b)  Multi-organizational  environment.  Approach  to  problem  solving  includes 
two  levels  (Coordinating;  Collaborating).  Multi- organization  environment  includes  two  levels  of 
pod  composition  (Homogeneous  organizations;  Mixed  types  of  organizations),  and  two  levels  of 
inter-pod  interaction  (Closed  interaction;  Open  interaction). 

The  main  dependent  variables  are  problem  solving  processes  and  outcomes  such  as  decision 
quality,  satisfaction  with  problem  solving  process  according  to  actors  and  to  external  panel,  task 
cohesion,  participation,  time  and  agreement.  For  a  more  detailed  description  of  the  independent 
and  dependent  variables  see  the  previous  report  “Research  Using  In  Vivo  Simulation  of  Meta- 
Organizational  Shared  Decision  Making  (SDM):  Task  2:  Development  of  an  experimental  plan 
for  in  vivo  exercise  and  simulation”. 

3.2  Ethics  Approval 

Before  the  execution  or  testing  of  any  of  the  components  of  the  in  vivo  simulation  of  meta- 
organizational  shared  decision  making  project,  ethics  approval  had  to  be  granted.  Therefore, 
observing  the  guidelines  and  regulations  from  the  University  of  Ottawa  Ethics  Research  Board 
and  the  Tri-Council  Policy  Statement:  Ethical  Conduct  for  Research  Involving  Humans  (TCPS), 
the  ethics  application  involved  a  meticulous  and  arduous  effort  from  the  project  team  to  submit  a 
complete  application  under  the  minimal  risk  review  process  category  for  the  experimental  part, 
and  as  a  modification  for  a  research  project  for  the  qualitative  component. 

As  a  result,  the  University  of  Ottawa  Research  Ethics  Board  complying  with  the  Tri-Council 
Policy  Statement:  Ethical  Conduct  for  Research  Involving  Humans  (TCPS)  and  other  legislations, 
examined  and  approved  the  application  for  ethical  approval  of  the  Risk  Management  and 
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Governance:  Understanding  Problem  Solving  and  Decision  Making  (File  #  08-10-31),  granted  as 
of  September  14th  2010  (See  Annex  D  for  the  Ethics  Approval  Letter). 


3.2.1  Ethics  approval  process 

The  application  for  ethics  approval  was  submitted  as  a  minimal  risk  review,  which  is  used  for 
studies  that  pose  minimal  risk  to  the  participants  rights  and  safety,  and  that  comply  with  the 
principles  of  free  and  informed  consent,  privacy  and  confidentiality,  appropriate  balance  of  risks 
and  benefits,  between  others.  The  ethics  application  addressed  research  protocols,  methodologies 
and  processes,  recruitment  and  selection  of  participant’s  processes,  benefits  and  risks  posed  to  the 
participants,  the  consent  process,  as  well  as  the  privacy,  confidentiality  and  anonymity  of  the 
participants. 

Consequently,  the  ethics  application  included  the  submission  of  the  following  documents  {Annex 
E,  &  F),  for  both  the  qualitative  and  quantitative  components: 

•  An  invitation  email  for  recruitment 

•  A  participant  consent  form 

Specifically  the  qualitative  component  required  the  submission  of  the  qualitative  interview  guide 
(See  Annex  A),  while  the  quantitative  in  vivo  simulation  experiment  required  the  participants’ 
questionnaires  (See  Annex  G ),  and  the  description  of  the  scenario  injects  (Please  refer  to  Section 
3.6). 

3.2.2  Data  storage  and  participant  identifiers 

In  compliance  with  the  Tri-Council  Policy  Statement:  Ethics  Conduct  for  Research  Involving 
Humans  (2008,  p.44),  “The  duty  of  confidentiality  includes  obligations  to  protect  the  data  from 
unauthorized  access,  use,  disclosure,  modification,  loss  or  theft”.  Different  procedures  were 
developed  to  ensure  that  all  data,  including  written  records,  consent  forms,  questionnaires,  video 
and  audio  tapes  will  be  maintained  in  a  locked  filing  cabinet  within  a  locked  office  within  the 
secure  zone  of  the  Faculty  of  Social  Sciences  research  centre  at  DMS  3rd  floor.  All  electronic 
files  (including  digital  audio  and  video  recordings)  with  participant  data  are  being  encrypted  and 
password  protected.  The  files  will  be  stored  on  a  dedicated  server  to  which  only  the  research  team 
has  access.  The  data  will  be  conserved  for  a  period  of  10  years  once  the  relevant  publications 
have  been  produced.  During  the  conservation  period,  all  hard  copy  data  (e.g.,  questionnaires)  will 
be  stored  in  a  locked  filing  cabinet  at  the  University  of  Ottawa.  Electronic  files  will  remain 
encrypted,  password  protected  and  stored  on  a  server  to  which  only  the  research  team  has  access. 

Once  the  conservation  period  is  over,  the  hard  copy  data  will  be  shredded  and  the  electronic  files 
will  be  deleted  via  secure  deletion  methods. 

Each  participant  will  be  labelled  with  a  different  participant  identifier  to  ensure  participant 
confidentiality  and  proper  tracking  of  data.  Participant  identifiers  consist  of  five  numbers:  the  first 
two  numbers  indicate  the  session  number,  the  third  number  indicates  the  session  type  (1- 
coordination,  2-  collaboration),  the  fourth  number  indicates  the  pod  number,  and  the  fifth  and  last 
number  indicates  the  participant  number  (up  to  three  participants  per  pod).  For  example,  the 
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number  “05112”  indicates  this  participant  is  part  of  session  five,  is  in  a  coordination  session,  is 
part  of  pod  one  and  is  participant  two  in  this  pod.  These  identifiers  will  be  used  to  label  each 
participant’s  documents.  These  documents  include:  participant  consent  forms  (see  Annex  F), 
background  questionnaires  (see  Annex  H),  confidentiality  and  intellectual  property  agreements 
(see  Annex  I),  participant  identification  and  information  linking  forms  (see  Annex  J),  task 
worksheets  (see  Annex  K),  task  questionnaires  (see  Annex  G),  and  consistency  of  interpretation 
questionnaires  (see  Annex  L). 

These  documents  will  be  stored  in  a  secure  environment  indicated  in  Section  3.9.6  Data 
Collection  and  Storage.  Data  that  contains  participant’s  names,  such  as  consent  forms, 
background  questionnaires,  confidentiality  and  intellectual  property  agreements,  and  participant 
identification  and  information  linking  forms,  will  be  stored  in  a  different  location  than  the 
participants’  worksheets  and  questionnaires  to  avoid  linking  data  with  individual  participants. 

3.3  Determining  scenario  complexity 

The  PODS  experiment  is  designed  to  measure  the  impact  that  the  approach  to  multi- 
organizational  decision-making  and  the  types  of  multi-organizational  environments  have  on 
problem  solving  processes  and  outcomes  during  complex  extreme  events.  Thus,  the  experimental 
design  involved  holding  situational  complexity  constant  as  a  control  variable,  while  the  following 
independent  variables  were  manipulated:  Approach  to  problem  solving  (collaboration  or 
coordination),  as  well  as  Multi-organizational  environments  (homogeneous  or  mixed;  and  open  or 
closed). 

In  keeping  with  the  experimental  design,  the  simulation  needed  to  reflect  a  high  level  of 
complexity  characteristic  of  an  extreme  event.  This  was  achieved  by  keeping  the  factors  and 
elements  that  contribute  to  situation  complexity  at  the  forefront  of  the  writing  process.  These 
factors  and  elements  are  a  part  of  the  Model  for  Inter-Organizational  Problem  Solving  devised 
during  Task  1  of  this  project  (see  Figure  4  below  for  a  breakdown  of  these  factors  and  elements 
within  the  model).  Rating  scales  were  subsequently  developed  and  utilized  by  expert  reviewers  to 
ensure  that  a  complex  event  was  indeed  being  portrayed  in  the  simulation. 
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Figure  4:  Factors  and  elements  contributing  to  situation  complexity 


3.3.1  Scenario  development  process 

Three  separate  scenarios  (see  Annex  M)  were  developed  for  the  scenario  assessment  process.  A 
detailed  simulated  city  was  used  to  situate  these  scenarios.  This  fictional  city,  entitled  Gapville,  is 
a  mid-sized  Canadian  city,  located  close  to  an  international  border,  and  features  all  of  the 
requisite  infrastructure,  populations,  and  development  to  provide  a  simulation  with  a  high  degree 
of  realism.  This  intricate  backdrop  became  the  backbone  for  three  simulation  storylines:  1)  a  train 
derailment  and  chemical  contamination  scenario;  2)  a  cyber  attack  and  blackout  scenario;  and,  3) 
a  radiological  ‘dirty  bomb’  scenario.  All  three  scenarios  were  written  using  peer-reviewed 
research,  salient  technical  reports,  and  existing  federal,  provincial,  and  local  emergency  plans  to 
guide  the  accuracy  of  the  events  in  the  scenarios. 


3. 3. 1.1  Scenario  design  and  selection  considerations 

Three  main  design  considerations  guided  the  development  of  the  scenarios  -  (1)  degree  of 
complexity,  (2)  accuracy,  and  (3)  plausibility  were  paramount  considerations  not  only  in 
constructing  the  scenarios,  but  also  in  the  assessment  and  selection  process.  The  importance  of 
designing  a  scenario  with  a  high  degree  of  complexity  has  already  been  noted,  but  accuracy  and 
plausibility  were  also  essential  design  considerations.  Accuracy  was  wanted  (especially  in 
technical  or  scientific  areas)  since  it  adds  to  the  realism  of  the  scenario  and  encourages 
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participants  to  engage  with  the  content  and  not  “fight”  it.  Disagreements  among  participants 
about  technical  details  may  cause  them  to  lose  focus  on  the  tasks  to  be  completed.  Also,  given  the 
in  vivo  aspect  of  the  study,  the  scenario  selected  must  have  the  potential  to  actually  occur. 
Scenario  plausibility  was  needed  for  participants  to  willingly  suspend  their  disbelief  during  the 
experiment. 

3.3.2  Scenario  assessment  process 

For  this  experiment,  three  scenarios  were  developed  and  assessed  using  focus  groups  and 
interviews  held  with  experts  in  the  field  of  emergency  management.  The  scenarios  were  validated 
for  accuracy,  plausibility  and  degree  of  complexity  using  a  scenario  rating  assessment 
questionnaire.  The  scenarios  assessed  included  (1)  a  train  derailment  and  subsequent  chemical 
contamination  scenario;  (2)  a  cyber  attack  scenario  occurring  in  conjunction  with  a  blackout  and 
extreme  winter  weather;  and  (3)  a  radiological  attack,  or  ‘dirty  bomb’  scenario. 

3.3.2. 1  Identification  and  selection  of  reviewers 

Raters  of  scenarios  were  screened  to  meet  a  set  of  inclusion  criteria  (see  Table  1).  Reviewers  were 
chosen  based  on  their  expert  level  knowledge  of  emergency  management  and  their  membership  in 
organizations  that  respond  to  or  manage  extreme  events.  More  than  fifteen  invitations  were  sent 
to  potential  participants.  Eight  experts  (seven  males  and  one  female)  participated  in  the  expert 
rating  session  held  at  the  University  of  Ottawa  on  October  17th,  2010  (see  Table  2  for  a  list  of 
participant  organizations).  Additional  meetings  were  held  with  participants  on  an  individual  basis 
as  schedules  allowed.  Participating  organizations  included  Federal,  Provincial  and  Municipal 
agencies  responsible  for  health,  safety,  security  and  defence  tasks.  Data  from  the  rating 
questionnaires  were  entered  into  SPSS  and  frequencies  were  analyzed. 

Table  1:  Inclusion  criteria  for  selecting  reviewers 


Inclusion  Criteria 

1. 

Expert  knowledge  of  emergency  management  and  response  to  extreme  events 

2. 

Member  of  an  organization  involved  with  emergency  response  or  management 

3. 

Able  and  willing  to  volunteer  to  participate  in  the  study 

4. 

Able  and  willing  to  participate  in  English 

Table  2:  List  of  Participant  Organizations 


Sector/Type  of  Organization 

Organizations 

Other  Federal  Government 
Departments  and  Agencies 

•  Public  Safety 

•  Canadian  Forces 
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Provincial  Government 
Departments  and  Agencies 

•  Office  of  Disaster  Management,  Manitoba  Health 

Municipal  Government 
Departments  and  Agencies 

•  Ottawa  Public  Health 

•  Ottawa  Fire  Department 

•  City  of  Ottawa 

3. 3.2. 2  Scenario  rating  assessment  questionnaire 

A  Scenario  Assessment  Guide  (see  Annex  M)  was  developed  to  guide  raters  through  the  scenario 
assessment  process.  The  reviewers  were  asked  to  complete  a  consent  form  and  a  confidentiality 
form  at  the  outset  of  the  rating  session.  Next,  raters  were  given  some  background  information  on 
the  project  and  given  instructions  on  how  to  fill  out  the  rating  instrument.  Prior  to  beginning  the 
assessment  of  the  scenarios,  these  raters  also  filled  out  a  Background  Questionnaire  (see  Annex 
M.3)  which  contained  questions  of  a  socio-demographic  nature.  The  expert  reviewers  then  read 
over  the  three  written  scenarios  and  filled  out  the  Scenario  Assessment  Questionnaire  (see  Annex 
M.6.M.8  &M.10). 

The  Scenario  Assessment  Questionnaire  was  comprised  of  22  questions  designed  to  capture  a 
reader’s  understanding  of  what  contributes  to  the  complexity  of  a  given  scenario.  All  three 
scenarios  were  assessed  using  the  same  questionnaire.  First,  reviewers  were  asked  to  rate  each 
scenario  as  ‘simple’,  ‘complicated’  or  ‘complex.’  Potential  contributions  to  complexity  for  a 
scenario  were  described  in  terms  of  impact,  uncertainty  and  vulnerability,  with  specific 
challenges  or  areas  noted  for  each.  After  reading  the  scenario,  the  reviewers  were  then  asked  to 
rate  the  degree  to  which  each  challenge  was  evident  in  the  scenario.  Answers  were  given  based  on 
a  5 -point  Likert  scale  ranging  from  0  (not  at  all  evident)  to  4  (strongly  evident)  for  each  potential 
contribution.  Then  reviewers  were  asked  which  of  the  factors  contributed  most  to  the  level  of 
complexity  of  a  given  scenario  and  to  explain  their  choice  of  answer.  Flere,  reviewers  chose 
between  ‘impact’,  ‘uncertainty’,  ‘vulnerability’  or  ‘all  of  the  above.’  Reviewers  were  also  asked 
to  rate  the  overall  level  of  complexity  of  the  scenario  on  a  Likert  scale  of  0  (not  at  all  complex)  to 
4  (very  complex).  Finally,  reviewers  rated  their  level  of  agreement  with  the  plausibility  of  the 
scenario  on  a  scale  of  0  (strongly  disagree)  to  4  (strongly  agree).  The  results  were  compared  with 
data  collected  from  other  reviewers  to  select  a  suitably  complex  scenario  for  use  in  the  simulation 
exercise. 

Focus  groups  and  meetings  with  individual  reviewers  also  involved  unstructured  discussion  of  the 
scenarios.  This  generated  valuable  feedback.  These  discussions  allowed  experts  to  address 
specific  issues  of  technical  and  scientific  accuracy  within  the  scenarios,  and  to  describe  positive 
and  negative  aspects  choosing  one  scenario  over  another  based  upon  their  own  experience  in 
responding  to  extreme  events. 

3.3.3  Scenario  assessment  results 

The  scenario  assessment  results  include  focus  group  comments,  written  commentary  from  the 
Scenario  Assessment  Guide,  and  the  results  from  the  scenario  assessment  questionnaire. 
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3.3.3. 1  Focus  group  comments  and  observations  on  scenarios 

At  the  end  of  the  focus  group  session,  reviewers  indicated  that  the  most  complex  scenario  was  the 
radiological/nuclear  Mail  Scenario,  followed  by  the  Train  Scenario,  and  lastly  the  ice  storm  and 
cyber  attack  Blackout  Scenario.  In  addition,  reviewers  stated  that  the  final  scenario  should  be  one 
in  which  many  actors  can  have  credible  roles. 

Focus  Group  Comments  on  the  Blackout  scenario 

Reviewers  felt  that  the  cyber  attack  and  the  ice  storm  were  completely  separate  streams  of  activity 
in  terms  of  response.  One  reviewer  thought  this  scenario  was  reasonably  complex,  while  another 
felt  we  had  underplayed  the  impact  on  critical  infrastructure.  Another  reviewer  thought  this 
scenario  was  not  realistic  enough  and  stressed  that  the  military  would  be  of  last  resort  only. 

Focus  Group  Comments  on  the  Mail  Scenario 

Overall,  raters  felt  that  the  scenario  was  very  complex.  Reviewers  cited  a  number  of  factors 
contributing  to  the  high  level  of  complexity  in  the  scenario.  These  included: 

•  The  closure  of  the  hospital,  which  would  be  a  major  event  causing  many  ripples  province 
wide 

•  The  issue  of  public  trust 

•  The  emergence  of  “for-profit  experts”  (e.g.  sale  of  Prussian  blue  online) 

•  Containment  issues  from  the  blown  out  window  from  the  initial  explosion 

•  Cross-jurisdictional  issues 

•  The  arrival  of  international  media 

•  Public  affairs  and  communication  issues 

•  Response  would  be  125%  reactive  and  still  not  get  it  right  and,  as  a  result,  the  intelligence 
community  will  be  accused  of  failure. 

Suggestions  were  made  to  increase  the  level  of  complexity  further,  to  provide  an  even  greater 
opportunity  for  the  military  to  play  a  significant  role.  A  suggestion  was  made  by  one  reviewer  to 
make  the  decontamination  side  more  evident,  stating  that,  “it  really  becomes  a  logistics  and 
communications  nightmare  to  be  able  to  go  to  peoples'  homes  and  places  of  work  to  collect 
contaminated  clothing.”  It  was  also  suggested  that  the  second  event  in  this  scenario  would  not 
have  to  occur  to  add  to  the  complexity  of  the  situation.  A  threat  would  also  draw  resources  away, 
necessitating  more  help  from  other  levels  and  organizations. 

Raters  discussed  the  potential  actors  in  the  scenario.  These  comments  include: 

•  Once  the  municipality  cedes  control,  it  assumes  a  support  role,  posing  communication 
problems 

•  The  Province  would  have  to  make  an  official  request  to  the  Feds  for  help 

•  The  military  is  linked  in  as  soon  as  the  Province  communicate  with  the  Feds 
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•  NGOs  would  not  want  to  participate,  given  the  nature  of  the  event 

•  As  support  to  the  R.C.M.P.,  a  national  security  response  team  would  be  called  in 

The  timeline  of  the  scenario  was  questioned  by  reviewers.  Raters  stated  that  first  responders  (fire) 
would  know  quickly  that  it  was  a  radiological  event  as  they  wear  detection  gear  when  on  the 
scene.  Additionally,  it  was  felt  that  the  timeline  for  this  scenario  could  be  reduced  to  several 
hours  rather  than  several  days.  Reviewers  stated  that,  depending  on  the  type  of  event,  troops 
would  be  available  within  6,  12,  or  24  hours. 

Focus  Group  Comments  on  the  Train  Scenario 

One  reviewer  considered  this  scenario  to  be  complicated,  but  not  complex  because  it  did  not 
include  a  criminal  element  and  because  authorities  have  dealt  with  these  types  of  situations  before 
(though  perhaps  not  on  the  same  scale).  Reviewers  found  this  scenario  to  be  believable.  However, 
the  reviewers  requested  more  details  on  demographics,  critical  infrastructures,  existing  response 
capacity,  and  size  of  the  community  affected.  Overall,  the  two  biggest  issues  reviewers  saw  with 
this  scenario  were  public  health  and  the  clean-up  (i.e.,  consequence  management). 

The  timing  of  the  scenario  was  also  questioned.  Reviewers  stated  that  responders  (i.e.  fire)  would 
discover  quite  quickly  (under  an  hour)  what  the  chemical  was  and  set  up  a  perimeter  as 
prevention.  Furthermore,  reviewers  stated  that  the  Hazmat  team  would  arrive  on  scene  in  under 
an  hour  as  well.  Reviewers  from  Public  Safety  focused  on  national  and  international  resources. 
They  felt  that  the  local  community  would  declare  an  emergency  very  quickly  and  request 
assistance  and  the  military  would  provide  security.  Reviewers  envisioned  that  the  Province  would 
take  the  lead  at  first,  followed  by  Public  Safety  at  the  Federal  level  along  with  Health  Canada. 

A  suggestion  from  a  reviewer  was  to  make  this  scenario  more  complex  at  the  site  level  by  adding 
cars  containing  chemicals  that  would  react  with  the  breached  car.  Another  suggestion  for 
increasing  situational  complexity  was  to  add  a  national  security  issue  (i.e.  a  terrorist  cause). 

One  of  the  individual  meetings  yielded  an  important  observation  about  the  Train  Scenario, 
namely  that  while  the  content  of  the  scenario  may  be  accurate,  it  is  not  plausible  within  a 
Canadian  context.  The  problem  of  plausibility  the  reviewer  spoke  of  relates  to  the  fact  that  the 
chemical  being  transported  in  the  scenario  is  more  frequently  manufactured  and  used  on  site. 
Transportation  of  this  chemical  is  generally  avoided,  precisely  because  of  the  impacts  described 
in  the  written  scenario.  Thus,  it  was  felt  that  the  radiological/nuclear  scenario  was  more  plausible. 

3. 3. 3. 2  Written  comments  and  observations  on  scenarios 

A  number  of  the  reviewers  chose  to  write  comments  on  the  scenarios  in  the  margins  of  their 
Assessment  Guides.  These  comments  are  summarized  below. 

Written  Comments  on  the  Blackout  scenario 

Reviewers  indicated  that  the  timing  of  interventions  throughout  the  day  was  more  delayed  than 
they  would  be  in  reality.  For  instance,  one  reviewer  stated  that  hospital  overflow  would  be 
mitigated  at  an  earlier  time  than  indicated  in  the  scenario.  A  second  reviewer  pointed  out  that 
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action  on  the  part  of  the  local  community  networks  would  take  place  earlier  than  indicated.  A 
third  stated  that  it  took  too  long  to  realize  that  the  computers  did  not  work.  This  reviewer  also 
expected  the  power  restoration  to  be  outlined  as  a  priority  earlier  in  the  scenario.  In  addition,  one 
reviewer  thought  that  it  was  not  likely  for  the  community  center’s  generator  fuel  to  last  only  one 
night  and  that  it  would  be  more  likely  that  patients  would  be  moved  out  of  affected  areas  and/or 
discharged  into  family  care. 

Reviewers  requested  the  addition  of  more  specific  dates,  the  geographical  size  of  Gapville,  and 
the  population  size  of  Gapville.  Clarification  was  also  requested  with  respect  to  the  actors 
involved,  and  suggestions  were  provided  by  reviewers.  For  example,  one  reviewer  stated  that  the 
province  would  provide  aid  before  asking  for  resources  at  the  federal  level.  Another  reviewer 
viewed  the  armed  forces  as  a  last  resort  resource  that  was  not  necessary  in  this  scenario.  Another 
reviewer  disagreed  that  police  would  still  be  at  the  hospital  seven  days  after  the  initial  crisis. 
Finally,  one  reviewer  suggested  multiple  clarifications  on  the  figures  responsible  at  each  stage  of 
the  scenario.  For  example,  the  reviewer  would  prefer  the  phrase  “the  head  of  the  hospital”  to  be 
replaced  with  terms  such  as  “the  hospital  administrator”  or  the  “executive  director.” 

Written  Comments  on  the  Mail  Scenario 

Reviewers  indicated  issues  with  respect  to  timing  and  actors  involved  in  the  scenario.  More  detail 
in  terms  of  numbers,  scope,  capacity  and  demographics  were  also  requested. 

As  was  the  case  with  the  Blackout  Scenario,  there  was  a  request  for  clarification  in  terms  of  the 
responders  present.  For  example,  when  the  ‘dirty  bomb’  is  sent  to  the  television  station,  two 
reviewers  indicated  that  the  police  should  have  been  contacting  the  R.C.M.P.  and  not  the  military. 
One  reviewer  suggested  that  it  should  be  the  National  CBRN  Response  Team  that  should  be 
intervening  rather  than  the  HazMat  team  and  the  federal  investigators.  Furthermore,  this  reviewer 
remarked  that  the  military  would  not  be  capable  of  setting  up  decontamination  sites,  as  is  done  in 
the  scenario.  It  was  suggested  that  volunteer  organizations  be  used  to  help  handle  the  flow  of 
traffic,  enforce  the  evacuation  zone,  and  provide  shelter  and  support  for  displaced  persons.  In 
terms  of  identifying  the  peipetrators  responsible  for  the  dirty  bomb,  one  reviewer  suggested  that  it 
should  be  the  Feds  investigating  rather  than  the  local  police. 

The  timeline  was  also  mentioned  by  reviewers.  One  reviewer  felt  that  exposure  estimates  should 
have  been  given  much  earlier  in  the  scenario.  Another  reviewer  pointed  out  that  it  would  be 
helpful  to  know  the  month  and  year  the  scenario  occurred  in.  In  addition,  this  reviewer  felt  that 
the  timeline  was  too  long  and  unrealistic.  Another  reviewer  wanted  more  clarification  regarding 
the  amount  of  time  it  took  for  the  screening  and  treatment  process  at  the  hospital.  Two  reviewers 
stated  that  public  notification  should  occur  and  a  state  of  emergency  should  be  declared  during 
the  scenario.  Finally,  a  reviewer  noted  that  the  military  would  arrive  within  4  hours,  not  72. 

Written  Comments  on  the  Train  Scenario 

Reviewer  comments  indicated  that  they  would  like  more  specifics  about  the  scenario.  For 
instance,  specifics  were  requested  regarding  the  type  of  respirators  paramedics  wore  (two 
reviewers),  the  size  of  Gapville,  and  the  number  of  citizens  in  Gapville  (three  reviewers),  the 
number  of  people  living  in  the  1 1km  radius  of  the  evacuation  zone  (one  reviewer),  and  the 
weather  conditions  (one  reviewer).  Finally,  clarification  was  suggested  when  using  words  like 
“authorities”  to  indicate  who  exactly  these  authorities  were  (one  reviewer). 
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As  in  the  other  two  scenarios,  there  were  comments  regarding  delays  in  the  timeline  and  the 
responders  involved.  One  reviewer  suggested  that  international  assistance  should  be  requested, 
and  three  reviewers  pointed  out  that  the  involvement  by  the  Feds  should  occur  right  away  rather 
than  the  day  after  the  initial  crisis.  One  reviewer  pointed  out  that  a  state  of  emergency  should  be 
declared. 

3. 3. 3. 3  Quantitative  Analysis  of  Scenario  Ratings 

This  section  describes  the  quantitative  results  from  the  Scenario  Assessment  Questionnaire.  In 
terms  of  the  overall  level  of  complexity,  it  was  found  that  the  Train  Scenario  had  the  highest 
complexity  rating.  The  Mail  Scenario  was  the  second  most  complex  and  the  Blackout  Scenario 
was  the  least  complex  of  the  three.  See  the  table  below  for  the  mean  level  of  complexity  for  each 
scenario,  measured  on  a  Likert  scale  from  not  at  all  complex  (0)  to  very  complex  (4). 


Table  3:  Mean  Level  of  Complexity  by  Scenario 


Blackout 

Mai 

1 

Train 

Mean  (n) 

Mode 

Mean  (n) 

Mode 

Mean  (n) 

Mode 

Mean  Level  of  Complexity 

2.8  (5) 

2;4 

3.0  (5) 

3 

3.2  (5) 

4 

The  results  of  the  above  table  are  also  reflected  in  the  average  mean  of  the  complexity  factors, 
with  the  Train  Scenario  having  the  highest  mean  complexity  factor,  followed  by  the  Mail 
Scenario  and  the  Blackout  Scenario.  The  mean  of  the  impact  factor  was  highest  for  the  Train 
Scenario.  Uncertainty  factor  means  varied  little  across  the  three  scenarios,  with  the  Blackout 
Scenario  having  a  slightly  higher  mean  level  of  uncertainty.  Likewise,  vulnerability  factor  means 
did  not  vary  significantly,  though  the  vulnerability  mean  was  marginally  higher  in  the  Mail 
Scenario  and  Train  Scenario.  The  table  below  summarizes  the  key  findings  from  the  assessment 
questionnaire  on  the  factors  of  impact,  uncertainty  and  vulnerability,  providing  the  means  and 
modes  separated  by  scenario. 


Table  4:  Means  and  Modes  of  Factors  and  Elements  by  Scenario 


Blackout 

Mail 

Train 

Mean 

(n) 

Mode 

Mean 

(n) 

Mode 

Mean 

(n) 

Mode 

Average  Mean  of  Complexity 
Factors 

2.5 

N/A 

2.7 

N/A 

2.8 

N/A 

IMPACT 

Impact  Severity 

2.5  (6) 

2 

3.6  (7) 

4 

3.9  (7) 

4 

Impact  Scope 

3.2  (6) 

3 

3-4  (7) 

4 

3.9  (7) 

4 

Impact  Timing 

3.0(5) 

2;4 

3.0  (7) 

3 

3.6  (7) 

4 

Media  Involvement 

2.2  (5) 

2 

2.6  (7) 

2 

3-4  (7) 

4 

Political  Processes 

3.2  (5) 

3;4 

3.3  (7) 

3 

3.6  (7) 

4 

Mean  of  Impact  Factor 

2.8 

3.2 

3.7 

UNCERTAINTY 

Novelty  of  a  Situation 

3.0  (6) 

4 

2.9  (7) 

4 

2.4  (7) 

2 

Anticipation  and  Planning 

2.0  (6) 

2 

1.8(5) 

2 

2.5  (6) 

2;3 

Data  and  Information 

2.0  (5) 

2 

2.1  (7) 

i;2 

2.4  (7) 

2 
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New  Organizations  and  Partners 

2.4  (5) 

2 

1.7(6) 

2 

1.8(6) 

1 

Changing  Context 

3.0  (6) 

3 

3.0  (7) 

2;3;4 

2.9  (7) 

2 

Flexibility  of  Interpretive 

Frameworks 

2.4  (5) 

2 

2.3  (6) 

2 

2.0  (7) 

2 

Mean  of  Uncertainty  Factor 

2.5 

2.3 

2.3 

VULNERABILITY 

Economic  Development 

2.4  (5) 

2 

2.2  (6) 

l ;  2 

2.2  (6) 

i;2 

Social  Capital 

2.0  (6) 

2 

3.0  (7) 

4 

2.6  (7) 

3 

Community  Competence 

2.0  (5) 

2 

2.5  (6) 

2 

2.0  (7) 

2 

Information  and  Communication 

2.0  (5) 

2;3 

2.7  (7) 

2;  3;4 

2.9  (7) 

4 

Other  Infrastructure 

2.6  (5) 

2;4 

2.3  (6) 

2 

2.9  (7) 

2 

Mean  of  Vulnerability  Factor 

2.2 

2.5 

2.5 

**  Note:  (n)  is  the  number  of  item  respondents 


Quantitative  Analysis  of  the  Results  for  the  Blackout  Scenario 

Of  the  six  reviewers  who  responded,  half  (50%)  rated  the  overall  scenario  as  complicated,  where 
the  other  half  (50%)  rated  it  as  complex.  Reviewers  were  asked  which  factor  contributed  most  to 
the  level  of  complexity  in  the  Blackout  Scenario.  Of  the  six  reviewers  who  responded,  the 
majority  (three)  chose  impact,  two  chose  vulnerability,  one  chose  all  of  the  above,  and  none  chose 
uncertainty.  This  response  was  reflected  in  the  means  for  each  factor,  whereby  impact  had  the 
highest  mean  (2.8),  uncertainty  had  the  second  highest  (2.5),  and  vulnerability  had  the  lowest 
mean  at  (2.2).  Impact  scope  and  media  processes  had  the  highest  means  among  the  elements 
making  up  these  factors. 

Reviewers  were  also  asked  to  rate  their  level  of  agreement  with  statements  on  the  scenario’s 
realism  and  plausibility.  Of  the  four  reviewers  who  responded,  two  reviewers  were  neutral  and 
two  were  in  agreement  (one  agree,  one  strongly  agree)  with  the  statement  that  the  first  part  of  the 
scenario  was  realistic  and  plausible.  The  majority  (three)  were  neutral,  and  one  agreed  with  the 
statement  that  the  second  part  of  the  scenario  was  realistic  and  plausible.  Lastly,  all  four  of  the 
reviewers  who  responded  were  neutral  with  the  statement  that  the  timeline  part  of  the  scenario 
was  realistic  and  plausible. 

Quantitative  Analysis  of  the  Results  for  the  Mail  Scenario 

Of  the  five  reviewers  who  responded,  the  majority  (60%)  rated  the  Mail  Scenario  as  complex  and 
40%  rated  it  as  complicated.  Results  were  mixed  concerning  the  reviewers’  estimations  of  which 
factor  contributed  most  to  the  level  of  complexity  in  the  scenario.  One  reviewer  stated  that  it  was 
impact,  one  stated  uncertainty,  and  two  stated  vulnerability.  Two  reviewers  reported  all  of  the 
above;  indicating  that  they  felt  all  three  factors  influenced  the  assessment  equally. 

Results  were  mixed  in  terms  of  questions  on  realism  and  plausibility.  The  majority  agreed  that  the 
first  part  of  the  scenario  was  realistic  (two  strongly  agree,  one  agree  and  one  neutral).  One 
reviewer  strongly  agreed  that  the  second  part  of  the  scenario  was  realistic,  where  two  were  neutral 
and  one  disagreed.  Half  of  reviewers  (two)  strongly  agreed  that  the  timeline  for  the  scenario  was 
realistic  and  the  other  half  (two)  disagreed  with  this  statement. 
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Quantitative  Analysis  of  the  Results  for  the  Train  Scenario 

Of  the  six  reviewers  who  responded,  half  (50%)  rated  the  Train  Scenario  as  complicated  and  the 
other  half  (50%)  rated  it  as  complex.  Four  reviewers  reported  their  estimation  of  which  factors 
contributed  most  to  the  level  of  complexity  in  the  Mail  Scenario.  Two  chose  impact,  one  chose 
uncertainty,  and  one  chose  vulnerability  as  contributing  the  most  to  complexity. 

Nearly  all  reviewers  (four)  reported  that  they  agreed  that  the  first  part  of  the  scenario  was  realistic 
(two  strongly  agree  and  two  agree),  while  one  was  neutral.  Two  reviewers  strongly  agreed  that 
the  second  part  of  the  scenario  was  realistic  and  three  were  neutral.  Finally,  three  reviewers 
agreed  that  the  timeline  for  the  scenario  was  realistic  (one  strongly  agree  and  two  agree),  while 
two  remained  neutral. 

3. 3. 3. 4  Summary  of  findings 

Though  reviewers  found  the  train  derailment  scenario  to  be  the  most  complex  according  to  our 
rating  scales  (albeit,  not  by  a  large  margin),  reviewers  identified  the  radiological  scenario  as  most 
complex  when  asked  during  the  verbal  portion  of  the  assessment  process.  Notably,  prior  to 
completing  the  rating  scales,  reviewers  most  frequently  cited  the  radiological  scenario  as  most 
complex  when  asked  if  the  scenario  was  best  described  as  simple,  complicated  or  complex.  After 
completing  the  rating  scales,  reviewers  cited  the  train  derailment  scenario  as  the  most  complex 
when  measured  on  a  scale  of  not  at  all  complex  (0)  to  very  complex  (4).  Corresponding  results 
between  the  rating  scale  means  and  overall  complexity  level  means  indicate  some  degree  of 
promise  in  the  efficacy  of  the  scales.  The  conflicting  results  may  be  partially  explained  by  the 
difficulty  many  have  with  distinguishing  complicated  situations  from  complex  (as  reflected  by  the 
wording  of  the  first  question  on  describing  the  scenario  as  simple,  complicated  or  complex). 
Additionally,  these  conflicting  results  may  indicate  that  reviewers  are  taking  in  more  elements 
into  their  decision  after  reviewing  the  scales  than  they  did  before  filling  out  the  rating  scale. 

The  radiological  scenario  was  subsequently  chosen  because  of  the  number  of  organizations  that 
would  become  involved,  locally,  regionally,  federally  and  internationally.  In  addition  to  being 
highly  complex,  the  scenario  chosen  also  needed  to  plausibly  involve  the  Canadian  Forces;  given 
the  security  implications  of  a  dirty  bomb  scenario,  and  the  decontamination  knowledge  and 
capabilities  in  the  Canadian  forces,  it  was  determined  that  the  radiological  scenario  best  fit  the 
criteria. 

Disagreements  with  respect  to  the  timeline  and  plausibility  of  the  scenario  were  reflected  in  the 
discussion  groups  and  in  the  questionnaire  response,  thus  the  research  team  concentrated  its 
efforts  on  scenario  refinement  in  these  areas.  Additional  attention  was  also  given  to  making  the 
Mail  Scenario  even  more  complex. 

3.3.4  Rating  scale  refinement  for  future  research  purposes 

An  extended  version  of  the  rating  scale,  complete  with  sub-elements,  shows  promise  as  a  tool  for 
measuring  situation  complexity.  This  is  indicated  by  results  produced  to-date.  The  rating  scales 
also  have  the  potential  to  be  extended  for  other  uses.  With  further  refinement  and  testing,  these 
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scales  may  be  used  in  scenario  development  for  training  exercises  and  as  a  tool  for  retrospective 
case  study  analyses. 


3.4  Instrument  development 

A  set  of  instruments  was  developed  for  the  simulation  experiment.  These  instruments  were  used 
to  measure  the  dependent  variables,  to  provide  background  information  on  the  participants,  to 
measure  the  participants’  ability  to  recall  and  understand  simulation  materials,  and  to  measure  the 
quality  of  decisions  made  by  participants.  These  instruments  are  described  in  more  detail  below. 

3.4.1  Dependent  measure  instrument  development 

The  dependent  measures  instrument  (see  Annex  G)  consists  of  three  parts;  the  first  part  includes 
questions  on  individual  participation  and  perspectives.  The  second  part  is  concerned  with 
questions  on  dynamics  within  the  pods.  The  third  part  focuses  on  questions  concerning  dynamics 
between  pods.  The  third  portion  of  the  instrument  was  not  included  in  the  closed  pod  sessions. 
Questions  on  these  questionnaires  are  made  to  be  answered  on  a  five  point  Likert  scale,  from 
strongly  disagree  (0)  to  strongly  agree  (4),  where  participants  answer  the  level  to  which  they 
agree  with  the  statement. 

The  questions  measure  participants’  satisfaction  with  problem  solving  process,  level  of 
participation,  task  and  group  cohesion,  and  level  of  agreement.  Questionnaires  were  reviewed 
during  initial  pre-testing  and  pilot  testing  phases.  It  was  discovered  that  a  few  of  the  questions 
were  duplicated  on  task  one  and  task  two  questionnaires  when  they  did  not  require  duplication. 
These  questions  were  taken  out  of  the  task  two  questionnaires  where  they  had  been  duplicated. 

3.4.2  Background  questionnaire 

Participants  were  asked  to  complete  a  Background  Questionnaire  (see  Annex  H)  during  the 
briefing  session  of  the  experiment.  The  questionnaire  was  comprised  of  both  open  and  closed 
questions.  Participants  were  asked  to  provide  their  current  job  title,  the  length  of  time  they  have 
worked  in  their  position,  with  the  organization  and  in  emergency  management,  and  to  describe 
their  main  responsibilities  with  respect  to  planning  and  responding  to  emergencies.  Other  socio¬ 
demographic  information  was  collected  such  as  their  age,  gender,  language  competency  and  level 
of  education.  During  the  pilot  testing,  participants  were  assigned  roles  and  thus  filled  out  the 
Background  Questionnaire  as  actors.  During  the  senior  participant  sessions,  roles  will  be  assigned 
based  on  the  participant’s  current  position  and  background. 

3.4.3  Consistency  of  interpretation  instrument 

A  consistency  of  interpretation  questionnaire  (see  Annex  L)  was  developed  to  determine  the 
degree  to  which  the  participants’  interpretation  of  the  scenario  matched  the  information  delivered 
during  the  session.  The  score  on  this  questionnaire  demonstrates  whether  the  information 
delivered  to  the  participant  was  absorbed  and  retained.  It  may  also  reflect  the  participants’  level 
of  participation  with  respect  to  engaging  with  inject  materials.  The  consistency  of  interpretation 
instrument  was  designed  to  enquire  about  items  of  importance  to  emergency  managers.  Questions 
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were  specifically  developed  to  ask  about  items  that  would  be  useful  and  important  to  carry  out  the 
two  tasks  in  the  in  vivo  experiment.  The  questionnaire  consists  of  ten  questions  and  is 
administered  at  the  end  of  the  session  once  all  of  the  binders  have  been  collected.  It  is  distributed 
at  this  time  in  order  to  ensure  that  participants  are  answering  from  memory  and  not  from  the 
binder  materials. 

3.4.4  Decision  quality  rating  instrument 

One  key  dependent  measure  is  the  assessment  of  decision  quality.  The  decisions  developed  by 
pod  participants  via  the  coordinating  and  collaboration  tasks  were  assessed  by  a  small  group  of 
raters  (n=3)  using  a  guide  developed  to  collect  their  ratings  on  dimensions  of  decision  quality 
based  on  elements  derived  from  decision  analysis  theory  (Edwards,  Miles  &  von  Winterfeldt, 
2007),  and  the  Shared-Decision  Making  Framework.  Examples  of  elements  rated  include  use  of 
correct  logic,  indication  of  clear  preferences,  use  of  an  appropriate  frame,  appropriate  use  of 
resources,  and  creativity.  The  rating  scale  was  developed  using  a  5-point  Likert  scale  to  capture 
raters’  judgements  on  the  various  elements  for  each  decision  (see  Annex  AO- 

Given  the  diversity  in  expression  of  decisions  across  pod  members  and  pods,  the  study  team 
formatted  decisions  using  extractions  from  the  individual  worksheets  and  recordings  so  that  each 
decision  rated  was  of  similar  format  and  had  similar  levels  of  detail.  Once  decisions  were 
compiled,  they  were  randomly  sorted  for  each  rater,  with  the  rater  remaining  blind  as  to  whether 
the  decision  had  been  derived  from  a  collaboration  or  coordination  task.  A  rating  guide  was 
completed  by  each  rater  for  each  decision.  All  decisions  from  communication  tasks  were  grouped 
together  for  assessment,  as  were  the  decisions  from  the  health  and  safety  tasks.  Inter-rater 
reliability  was  assessed  using  Fleiss’s  kappa  and  intra-class  correlation. 

3.5  Development  and  assessment  of  simulation  tasks 

The  development  of  the  simulation  tasks  began  prior  to  the  scenario  assessment  process.  Draft 
versions  of  these  tasks  were  assessed  during  scenario  rating  sessions  and  individual  meetings  with 
expert  raters.  Following  the  scenario  selection  process,  the  radiological  scenario  was  developed 
for  the  simulation,  and  the  tasks  were  further  refined  to  reflect  specific  issues  of  concern  brought 
out  by  this  scenario,  such  as  health  and  safety  concerns.  This  section  of  the  report  describes  the 
links  between  the  simulation  tasks  and  the  Shared  Decision-Making  Model,  the  rationale  for 
assessing  simulation  tasks,  and  the  task  development  and  refinement  process. 

3.5.1  The  Shared  Decision-Making  Model  as  it  relates  to  simulation 
tasks 

The  experimental  design  involves  the  manipulation  of  two  levels  of  the  independent  variable, 
approach  to  problem-solving.  These  levels  are  coordination  and  collaboration.  These  two  levels 
are  depicted  on  the  right  hand  side  of  the  Model  for  Inter-Organizational  Problem  Solving,  which 
is  pictured  and  described  in  the  introduction  section  of  this  report.  In  order  to  manipulate  the 
implementation  of  these  problem-solving  approaches,  two  sets  of  tasks  were  developed  -  one  set 
of  tasks  to  elicit  coordinating  behaviour,  and  one  set  of  tasks  to  elicit  collaborating  behaviour. 
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In  addition  to  facilitating  the  manipulation  of  the  approach  to  problem  solving  variable,  the  tasks 
were  also  designed  to  limit  the  moderating  effects  of  problem  solving  stage  to  three  stages  of  the 
problem  solving  cycle  represented  in  the  figure  below.  Each  task  was  divided  into  three  sub-tasks, 
and  each  of  these  sub-tasks  related  directly  to  one  of  the  following  problem-solving  stages: 
problem  definition,  solution  generation,  and  decision  making. 


Figure  5:  Generic  stages  of  problem-solving 

3.5.2  Rationale  for  assessing  simulation  tasks 

Draft  tasks  were  reviewed  by  individuals  with  an  expert  level  of  knowledge  in  the  field  of 
emergency  management,  and  who  were  currently  members  of  organizations  involved  in 
emergency  response.  The  assessment  of  these  tasks  was  completed  through  meetings  and  sessions 
held  with  these  reviewers.  In  particular,  these  assessments  sought  feedback  with  respect  to 
relevancy  in  terms  of  the  problem  identified  in  the  task.  Validation  was  also  necessary  in  order  to 
ensure  that  the  tasks  elicited  the  desired  behaviour  (coordinating  or  collaborating)  from  the 
participants.  Reviewers  agreed  that  the  tasks  and  task  worksheets  elicited  the  desired  behaviour. 
Suggestions  were  made  by  the  reviewers  to  modify  some  issues  identified  in  the  tasks. 

3.5.3  Task  development  and  refinement  process 

The  final  tasks  were  developed  to  reflect  a  number  of  design  considerations.  These  considerations 
required  that  the  tasks: 

•  Elicit  the  desired  behaviour  and  approach  to  problem  solving  (coordinating,  collaborating) 

•  Include  three  sub-tasks  that  replicate  the  three  problem  solving  stages  targeted  in  the 
experimental  design  (problem  definition,  solution  generation,  and  decision  making) 
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•  Be  completed  within  45  minutes  to  60  minutes 

•  Address  issues  relevant  to  the  participants 

•  Be  plausible  within  the  selected  scenario 

Initially,  both  tasks  focused  on  communication  issues.  Expert  reviewers  suggested  that  one  task 
focus  on  either  evacuation  or  on  health  and  safety  issues  of  responders.  Health  and  safety  was 
later  chosen  following  the  selection  of  the  radiological  scenario,  as  this  theme  is  both  relevant  to 
study  participants  and  plausible  within  the  scenario. 

The  communication  tasks  and  the  health  and  safety  tasks  were  structured  using  task  worksheets, 
which  were  completed  by  participants  (see  Annex  K).  These  task  worksheets  were  developed  and 
worded  differently  to  elicit  either  coordinating  behaviour  or  collaborating  behaviour.  Task 
worksheets  were  also  adapted  for  closed  pods  in  order  to  reflect  the  lack  of  inter-pod 
communication  within  these  pods. 

3.6  Development  of  multimedia  simulation 

Following  the  scenario  and  task  selection  and  refinement  process,  the  mail  scenario  was  then 
transformed  into  a  multimedia  simulation  using  video,  animation,  audio  and  slide  presentation 
software.  The  outputs  from  the  multimedia  software  were  augmented  by  paper-based  materials  as 
well,  which  were  included  in  the  participant  materials  distributed  at  the  start  of  each  session.  This 
section  includes  the  design  considerations  for  the  multimedia  components,  the  techniques  used  to 
create  these  components,  and  finally,  a  description  of  the  multimedia  components  used  in  the 
Inter-GAP  In  Vivo  System. 

3.6.1  Design  considerations 

While  the  scenario  used  during  the  selection  and  refinement  process  was  designed  to  be  as 
detailed  as  possible  to  ensure  that  the  raters  were  able  to  assess  complexity,  some  details  were 
removed  in  the  final  multimedia  version  of  the  scenario.  This  was  particularly  the  case  with 
technical  aspects  of  the  scenario  that  could  be  considered  potential  areas  for  debate  and 
contention  among  participants  (e.g.,  the  first  responder  group  to  arrive  on  scene).  To  ensure  that 
the  participants  could  fully  engage  in  the  scenario  without  debating  unnecessary  details,  the 
scenario  was  reviewed  and  revised  to  include  only  those  details  which  potentially  would  have  a 
direct  bearing  on  the  tasks  required. 

The  introduction  of  situation  reports  and  news  reports  into  the  multimedia  components  served  to 
encourage  participants  to  fully  engage  with  inject  materials.  It  was  determined  that,  by 
introducing  familiar  formats  of  information  such  as  situation  reports,  news  reports  and  press 
releases  into  the  simulation  materials,  participants  would  become  more  fully  immersed  in  the 
scenario. 

Social  media  outputs  were  also  included  among  the  materials  that  participants  received  in  the 
final  version  of  the  mail  scenario.  Social  media  components  included  a  YouTube  clip,  a  series  of 
Facebook  posts,  as  well  as  a  number  of  Twitter  feeds  (see  Annex  O  for  Pods  Participant 
Workbook  Materials).  These  injects  were  used  to  add  complexity  to  the  types  and  amounts  of 
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information  that  the  participant  received.  It  required  participants  to  sort  through  the  information 
for  relevancy  and  reliability,  much  as  responders  are  required  to  do  during  extreme  events. 

3.6.2  Creating  the  simulation 

After  undergoing  validation  by  a  series  of  experts  in  emergency  management,  as  well  as  by  a 
number  of  radiological  and  military  experts,  the  ‘dirty  bomb’  mail  scenario  was  transformed  into 
multimedia  components.  Video  clips  were  completed  using  Xtranormal  animation  software, 
Adobe  Premiere,  and  stock  video  footage.  Social  media  feeds  and  posts  were  created  using 
Facebook  and  Twitter,  as  well  as  with  Adobe  Photoshop.  The  fictional  city  of  Gapville,  used  to 
situate  the  scenario,  was  created  using  Adobe  Illustrator.  Individual  Gapville  maps,  which  were 
part  of  the  participant  materials  handed  out  during  the  simulation,  were  also  created  with  the  aid 
of  Adobe  Photoshop  and  Adobe  Illustrator.  Microsoft  Word  was  used  to  create  the  Public  Health 
Agency  Press  Release  inject,  while  PowerPoint  was  used  to  create  the  instructions  for  the 
participants  that  were  delivered  via  NEFSIS  video  teleconferencing  software. 

3.6.3  Multimedia  components 

The  multimedia  components  are  described  in  this  section.  These  components  include  video  clips, 
the  illustrated  fictitious  city  of  Gapville ,  paper-based  injects,  and  PowerPoint  presentations. 

Video  Clips 

Video  clips  were  created  to  simulate  a  report  from  an  emergency  operations  centre  (EOC),  to 
simulate  news  reports  and  updates,  and  to  simulate  a  YouTube  rant  from  a  disgruntled  member  of 
the  public.  The  audio  tracks,  watermarks  and  the  ticker  tape  information  running  across  the 
bottom  of  the  screen  were  added  to  the  videos  using  Adobe  Premiere.  Some  of  the  clips  were 
augmented  with  stock  video  footage  that  was  purchased  for  a  nominal  fee  through  iStockphoto. 
Video  clips  were  animated  using  Xtranormal  Technology’s  video  software.  Voice  actors  were 
used  to  create  the  voices  for  animated  avatars,  which  took  the  place  of  live  actors  in  the  clips.  The 
voice  actors  were  casted  from  volunteers  from  the  University  of  Ottawa’s  theatre  program. 

Xtranormal’s  avatars  allowed  the  research  team  to  create  a  scenario  that  is  both  realistic  and  yet 
sufficiently  removed  from  reality  (in  that  the  characters  are  animated  rather  than  live  actors)  to 
reduce  the  potential  for  psychological  stress  in  participants.  This  software  also  allowed  a  high 
degree  of  flexibility  during  the  design  phase.  Its  use  was  both  more  efficient  and  more  cost- 
effective  when  compared  with  live  action  video  methods.  See  Figure  6  below  for  screenshots 
from  the  simulation. 
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Gapville 

The  mail  scenario  is  situated  in  the  fictional  city  of  Gapville,  a  highly  detailed  two-dimensional 
rendering  of  a  Canadian  city  located  on  the  border  to  the  United  States.  This  training  tool  was 
created  as  part  of  the  Psychosocial  Risk  Manager  ( PRiMer )  training  program,  funded  by  the 
Centre  for  Security  Sciences,  CRTI-06-0259TD  project.  Created  for  use  in  learning  activities  and 
tabletop  exercises,  this  cityscape  illustration  contains  all  the  built  and  social  infrastructure  of  a 
mid-sized  North  American  city.  Additionally,  the  city  of  Gapville  was  created  to  bring 
psychosocial  issues  to  the  forefront  of  emergency  management  training  exercises,  with  a  diverse 
range  of  at-risk  populations  depicted  alongside  other  populations  that  require  psychosocial 
consideration.  A  visual  representation  of  the  city  of  Gapville  can  be  seen  in  Figure  7. 
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Figure  7:  Simulated  city  of  Gapville  utilized  in  the  Inter-GAP  In  Vivo  System 


Paper-based  Injects 

A  press  release  from  the  Public  Health  Agency  of  Canada,  individual  maps  of  Gapville,  and 
outputs  from  simulated  Twitter  and  Facebook  posts  or  feeds  were  all  presented  in  a  paper-based 
format.  These  injects  were  included  as  part  of  the  package  of  information  given  to  each 
participant  at  the  start  of  the  simulation  session. 

PowerPoint  Presentations 

A  PowerPoint  presentation  was  used  to  deliver  the  instructions  to  participants  over  the  NEFSIS 
video  conferencing  system.  Given  that  instructions  varied  slightly  from  the  open  pods  to  the 
closed  pods,  two  versions  of  this  presentation  were  created.  The  slides  used  in  these  presentations 
can  be  seen  in  Annex  P. 

3.7  PODS  session  roles,  responsibilities  and  materials 

This  section  describes  POD  session  roles,  responsibilities  and  associated  materials  used  by  the 
facilitator,  controller,  observers  and  participants  during  the  course  of  a  complete  session  of  the 
simulation.  Many  of  the  specific  forms  and  instruments  used  in  the  simulation  have  been 
described  already  in  earlier  sections  of  this  report  and  are  only  referenced  here  as  appropriate. 
References  are  made  to  both  electronic  media  and  paper-based  materials  but  only  copies  of  paper- 
based  materials  are  included  in  the  appendices. 
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3.7.1  Facilitator  role  and  materials 

Facilitator  role 

The  facilitator’s  role  was  to  oversee  the  proper  conduct  of  each  session  of  the  experiment  as  per 
ethics  requirements.  The  facilitator  held  a  briefing  session  prior  to  the  experiment  followed  by  a 
debriefing  session  at  the  end  of  the  simulation.  The  facilitator  provided  participants  with  the 
study’s  agenda,  as  well  as  directions  on  how  to  complete  the  tasks.  An  explanation  of  the  study’s 
purpose  and  background  was  explained  in  more  detail  following  the  completion  of  the  study.  The 
facilitator  ensured  that  all  forms  and  materials  were  properly  administered,  completed  and 
collected  throughout  the  day. 

Briefing  materials 

The  following  briefing  materials  were  used  during  the  briefing  session  prior  to  the  start  of  the 
experiment: 

•  Information  Sheet  &  Consent  Form  (Refer  to  Annex  O  &  F) 

•  Confidentiality  and  Intellectual  Property  Agreement  (Refer  to  Annex  I) 

•  Participant  Background  Questionnaire  (Refer  to  Annex  H) 

•  A  short  introductory  presentation,  welcoming  participants,  explaining  their  role  and 
communication  options  within  the  pods  (Refer  to  Annex  P.  1  for  presentation  slide  deck.) 

The  briefing  session  lasted  about  half  an  hour  prior  to  study  commencement.  Participants  were 
invited  to  fill  out  consent  and  confidentiality  forms  as  per  ethics  requirements.  They  were 
informed  of  their  voluntary  participation  as  well  as  anonymity  procedures.  Each  participant  also 
filled  out  a  Participant  Background  Questionnaire.  Briefing  participants  on  the  schedule  and 
directions  for  the  experiment  allowed  for  questions  and  concerns  to  be  addressed  by  the 
facilitator.  This  ensured  transparency  of  participant  roles  and  responsibilities  expected  during  the 
experiment. 

Upon  completion  of  the  briefing,  participants  were  assigned  to  a  POD  and  taken  to  separate 
rooms.  When  in  the  separate  rooms,  each  participant  received  a  binder  with  more  background 
information  about  the  experiment  and  their  role.  In  line  with  anonymity  and  confidentially  ethics 
guidelines,  participants  were  assigned  a  number  when  given  their  participant  workbook. 
Participants  were  then  instructed  to  identify  themselves  on  video  by  the  number  assigned  them  for 
tracking  purposes.  A  hardcopy  of  the  Gapville  map  was  provided  to  each  POD  member  with  a 
Gapville  profile  and  demographics  table  inserted  into  their  participant  workbook.  The  complete 
contents  and  structure  of  the  participant  workbook  are  described  below  in  Section  3.7.4 
Participant  Materials.  A  short  video  depicting  the  Gapville  profile  and  its  background 
information  was  sent  to  each  pod  via  video  conferencing  software  (refer  to  Section  3.6 
Development  of  Multimedia  Simulation  for  more  information  about  the  multimedia  components 
of  the  simulation). 

Debriefing  materials 

The  following  materials  were  used  during  the  debriefing  session: 

•  Consistency  of  Interpretation  Instrument  (Refer  to  Annex  L) 
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•  Two  video  clips  -  an  overview  of  the  Model  of  inter-organizational  problem  solving  and  a 
summary  of  the  research  objectives  and  experimental  design. 

The  debriefing  session  lasted  half  an  hour.  Participants  were  asked  to  complete  the  Consistency  of 
Interpretation  instrument.  The  facilitator  then  showed  participants  the  two  video  clips. 
Participants  provided  feedback  and  were  able  to  ask  questions  regarding  the  study.  The  facilitator 
ensured  that  all  forms  and  binders  were  handed  in. 

3.7.2  Controller  role  and  materials 

Controller  role 

The  controller  was  responsible  for  running  the  video  conferencing  software  and  guiding  the 
participants  as  the  scenario  unfolded.  The  controller  warned  participants  of  time  limits  on  tasks, 
informed  them  when  communication  between  PODS  was  open  or  closed  and  directed  them  to  the 
Nefsis  chat  function  and  CB  radios. 

Simulation  delivery >  script 

The  controller  referred  to  a  script  (see  Annex  P.2 )  outlining  the  duties  and  procedures  for 
delivering  the  simulation.  These  duties  related  to  technical  issues  (e.g.,  plugging  in  headsets, 
recording,  audio  adjustments,  etc.),  sequencing  media  delivery  (e.g.,  scenario  videos,  power  point 
slides,  etc.),  layout  (i.e.  the  way  participants  viewed  themselves  in  the  conferencing  software  and 
the  size  of  the  slideshow  and  video  feeds),  and  voice  over  script  for  time  warnings  and 
instructions.  Included  in  this  script  was  a  breakdown  of  activities,  including  the  time  and  slide 
that  belonged  with  them. 

3.7.3  Observer  role  and  materials 

During  each  pilot  session,  a  number  of  observers  were  present.  The  observer’s  role  was  created  to 
oversee  the  distribution  and  collection  of  participant  materials  and  forms,  as  well  as  to  ensure  the 
linking  of  participants’  identification  numbers  with  the  written  materials,  forms,  and  audio  and 
video  recordings  from  the  in  vivo  sessions. 

Prior  to  the  start  of  each  session,  questionnaires  and  worksheets  included  in  the  participant 
binders  were  labelled  with  participant  ID  numbers  using  “sticky  labels”.  During  the  initial 
briefing,  as  participants  handed  in  their  consent  forms,  they  received  a  binder  of  participant 
information  and  questionnaires  labelled  with  their  participant  ID  number.  Four  loose  identifier 
labels  located  at  the  front  of  the  binder  were  removed  as  each  participant  received  their  binder, 
which  were  then  affixed  to  the  corresponding  forms  for  that  participant.  In  order  to  identify 
participants  with  the  audio  and  video  recordings,  participants  were  asked  to  address  both  the 
video  camera  and  the  web  camera  and  state  their  name,  organization,  and  participant  ID  number 
prior  to  the  start  of  each  session.  These  statements  were  made  following  the  briefing  session,  once 
participants  had  settled  into  their  separate  pod  rooms,  but  prior  to  the  start  of  the  simulation 
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3.7.3. 1  Observation  forms 


During  each  session,  a  number  of  forms  were  completed  by  the  observers  to  track  administrative 
information  and  observations  about  each  session,  pod  and  participant.  The  three  forms  developed 
and  implemented  were: 

•  Session  Administration  Information  Form 

•  Participant  Identification  and  Information  Linking  Form 

•  Technical  and  Process  Issues  Form 

Session  Administration  Information  Form 

This  form  was  completed  for  each  session  (one  form  per  session).  Once  the  briefing  session  had 
been  completed  and  the  simulation  was  underway,  an  observer  was  tasked  with  capturing  any 
information  that  could  not  be  completed  prior  to  the  start  of  the  session  (e.g.,  number  of 
participants  present,  number  of  pods,  etc.).  It  was  used  to  record  basic  information  about  the 
session,  including:  date,  session  number,  session  type  (university  students,  professional  students, 
or  senior  officials),  task  type  (collaboration  or  coordination),  pod  status  (connected  or  closed), 
number  of  participants  per  pod,  pod  type  (mixed  or  homogeneous),  scenario  used,  location,  the 
observer’s  initials,  the  facilitator,  the  start  time  and  end  time  of  the  session,  as  well  as  the  total 
runtime  for  the  session.  This  form  was  then  used  to  link  the  data  for  these  variables  to  each 
participant  record  in  the  SPSS  database,  where  appropriate.  (Please  see  Annex  P.3  for  a  copy  of 
the  Session  Administration  Information  Form.) 

Participant  Identification  and  Information  Linking  Form 

This  form  was  completed  for  each  participant  (one  form  per  participant).  Once  the  briefing 
session  had  been  completed  and  the  simulation  was  underway,  an  observer  was  provided  with  the 
completed  participant  consent  forms  collected  during  the  briefing  phase,  and  was  tasked  with 
completing  the  relevant  participant  form  (matched  according  to  labels  on  the  Consent  Form,  the 
Confidentiality  and  Intellectual  Property  Agreement  Form,  the  Background  Questionnaire,  as 
well  as  the  Participant  Identification  and  Information  Linking  Form).  The  Participant 
Identification  and  Information  Linking  Form  was  designed  to  capture  the  following  participant 
information:  participant’s  name,  the  four  digit  participant  ID,  the  session  number,  session  type 
(University  of  Ottawa  students,  professional  students  or  senior  officials),  the  participant’s  pod 
number,  the  participant’s  number  within  the  pod,  the  participant’s  organization,  the  participant’s 
organization  type  (military,  ICS  oriented,  and  non-ICS  oriented),  the  participant’s  pod  status 
(connected  or  closed),  as  well  as  the  number  of  participants  in  the  pod.  (Please  see  Annex  J  for  a 
copy  of  the  Participant  Identification  and  Linking  Form.) 

Technical  and  Process  Issues  Form 

This  form  was  completed  for  each  session  (one  form  per  session).  Observation  forms  were  used 
during  the  sessions  to  record  any  technical  or  process  issues  that  had  the  potential  to  affect  the 
data  quality  of  the  session.  The  technical  issues  form  consisted  of  questions  regarding  the 
presence  of  audio  or  video  delivery  problems,  audio  or  video  recording  problems,  inter-pod 
communication  problems  due  to  technology,  conferencing  software  problems,  internet  connection 
failures,  and  a  section  for  other  unanticipated  technical  issues.  The  process  issues  form  was 
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concerned  with  the  following  problems:  participant  absenteeism,  participant  lateness,  participants 
leaving  early,  consent  refusal,  delayed  start  time,  session  interruption,  participant  confusion,  and 
other  unanticipated  process  issues.  In  both  the  technical  issues  form  and  the  process  issues  form, 
there  were  prompts  to  describe  the  nature  of  the  problem(s)  in  more  detail.  (Please  see  Annex  P.4 
for  a  copy  of  the  Technical  and  Process  Issues  Form.) 

3.7.4  Participant  materials 

Participant  workbooks  were  prepared  for  use  by  participants  for  each  of  the  four  types  of  sessions 
conducted.  Tasks,  questionnaires  and  worksheets  provided  to  participants  varied  slightly 
depending  on  the  task  type  (coordination  or  collaboration)  and  the  pod  status  (open  or  closed). 
Below  are  the  four  types  of  participant  materials: 

1 .  Open  /  Coordination 

2.  Open  /  Collaboration 

3.  Closed  /  Coordination 

4.  Closed  /  Collaboration 

Refer  to  Section  3.5  for  a  detailed  description  of  the  types  of  tasks.  Annex  G  provides  examples  of 
the  questionnaires  differentiated  by  task  type  and  pod  status.  Annex  K  provides  examples  of  the 
task  worksheets  differentiated  by  task  type  and  pod  status. 

The  briefing  materials  contained  in  the  workbooks  are  similar  across  all  four  types  of  sessions. 
Also  similar  across  all  four  types  of  sessions  are  the  hard  copy  versions  of  the  situation  reports 
and  associated  press  releases  and  social  media  feeds.  A  debriefing  sheet  was  provided  containing 
the  contact  information  of  the  research  team  should  participants  have  any  questions  about  the 
simulation  at  a  later  date.  The  sequence  of  material  found  within  each  participant  workbook  is 
listed  below. 

•  Gapville  Profile 

•  Situation  Report  #1+  Public  Health  Agency  of  Gapville  Press  Release  Inject  * 

•  Task  1  +  Task  1  Questionnaire  * 

•  Situation  Report  #2  +  Facebook  Post  Inject  +  Twitter  Feeds  Inject  * 

•  Task  2  +  Task  2  Questionnaire* 

•  Debriefing  Page 

Materials  marked  with  an  asterisk  were  put  into  separate  envelopes  within  each  participant  binder 
and  opened  as  per  simulation  procedure.  Annex  O  contains  an  example  of  only  similar  materials 
across  all  session  types.  Annex  K  contains  the  different  task  types  and  Annex  G  the  different  types 
of  task  questionnaires. 
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3.7.5  Technical  and  logistical  support 


On  the  day  of  each  session,  a  technician  was  responsible  for  setting  up  each  room  as  per  section 
3.9  of  this  report.  The  technician  ensured  that  the  POD  rooms  and  the  control  room  were  properly 
set  up  and  that  the  video  conferencing  software  was  functioning  properly  and  accessible.  This 
person  was  on  hand  throughout  each  session  to  monitor  in  case  any  problems  occurred.  Logistical 
support  included  arranging  refreshments  for  participants  and  making  sure  appropriate  signage  was 
put  up  to  direct  participants  to  the  study. 


3.8  Recruitment 

Recruitment  efforts  centred  on  finding  three  types  of  participants:  experts  for  scenario  and  task 
assessment  as  well  as  for  decision  quality  rating;  naive  participants  and  junior  professionals  for 
pilot  testing;  and  senior  level  emergency  management  professionals  for  the  full  scale  experiment. 
Participant  involvement  in  the  study  was  completely  voluntary  and  consent  was  obtained  from 
each  participant  as  per  ethics  requirements. 

Recruitment  of  Experts 

Experts  were  needed  for  the  assessment  of  the  tasks  and  scenarios,  for  the  senior  level  re-review 
session,  and  for  the  decision  quality  rating  session.  Nine  experts  were  recruited  for  the  scenario 
and  task  assessments.  These  experts  provided  valuable  feedback  on  the  accuracy,  plausibility  and 
complexity  levels  of  the  scenarios  drafted  for  the  scenario  selection  process.  Expert  participants 
were  required  to  be  willing  to  participate  in  the  study,  able  to  participate  in  English,  and  able  to 
attend  scheduled  meetings  or  sessions.  An  expert  level  of  knowledge  of  emergency  management 
and  response  to  extreme  events  was  also  required.  See  the  table  below  for  specific  inclusion  and 
exclusion  criteria  for  this  group. 


Table  5:  Inclusion  and  Exclusion  Criteria  for  Expert  Recruitment 


Inclusion  and  Exclusion  Criteria 

Scenario  and  Task 
Assessments 

Senior 

Level 

Demo 

Decision 

Quality 

Rating 

Expert  knowledge  of  emergency  management  and  response  to  extreme 
events 

■A 

S 

Able  and  willing  to  volunteer  to  participate  in  the  study 

S 

Able  and  willing  to  participate  in  English 

s 

S 

S 

Outside  commuting  distance  of  the  study  site 

S 

S 

Member  of  an  organization  involved  with  emergency  response  or 
management 

s 

s 

Has  been  in  management-level  decision  making  role  for  his/her 
organization  for  at  least  12  months  (or  has  was  in  a  management-level 
decision  making  role  for  his/her  organization  for  at  least  12  months 
prior  to  occupying  a  new  position  or  prior  to  retirement) 

s 
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Recruitment  for  Pre-testing  and  Pilot  Sessions 


Naive  participants  were  required  for  the  pre-testing  and  pilot  testing  sessions.  Students  and  naive 
participant  volunteers  provided  valuable  feedback  related  to  the  content  and  process  of  the 
simulation.  Junior  level  career  professionals  and  students  engaged  in  emergency  management 
programs,  the  military  and  non-governmental  organizations  were  targeted  for  the  junior  level  pilot 
sessions.  These  sessions  provided  valuable  feedback  on  the  instruments  used  in  the  experiment, 
on  session  materials  and  on  the  development  of  coding  schemes.  Inclusion  and  exclusion  criteria 
for  this  groups  of  participants  is  shown  below. 

Table  6:  Inclusion  and  Exclusion  Criteria  for  Pre-testing  and  Pilot  Sessions 


Inclusion  and  Exclusion  Criteria 

Internal 

Pre-testing 

Sessions 

Naive 

Participant 

Pilot 

Sessions 

Professional 

Students/ 

Junior 

Career 

Sessions 

Able  and  willing  to  volunteer  to  participate  in  the  study 

P 

P 

U 

Able  and  willing  to  participate  in  English 

P 

P 

U 

Outside  commuting  distance  of  the  study  site 

P 

P 

U 

Student,  volunteer  or  team  member 

P 

University  student  or  volunteer  naive  to  the  purpose  of  the 
study 

P 

A  student  in  a  professional  emergency  management  program  or 
a  junior  level  member  of  an  organization  involved  with 
emergency  response  (a  member  a  military,  ICS,  or  a  relevant 
non-govemmental  organization) 

U 

The  research  team  used  a  University  of  Ottawa  study  recruitment  program,  called  the  Psychology > 
Integrated  System  of  Participation  in  Research  (ISPR),  to  recruit  naive  participants  during  the 
pilot  testing  phase.  The  present  study  was  posted  on  this  system  and  time  slots  were  made 
available  to  first  year  psychology  students.  Refer  to  Annex  Q.l  for  a  screenshot  of  the  posting 
made  on  the  ISPR  system.  The  research  team  was  notified  by  email  when  a  student  signed  up  for 
a  time  slot.  Students  remained  anonymous  and  only  an  identification  number  was  provided.  Upon 
completion  of  the  study,  the  research  assistant  entered  the  ISPR  system  and  awarded  the  student 
their  participation  credits. 

During  the  pilot  testing  phase,  naive  participants  were  assigned  fictitious  roles  within  the 
scenario,  as  preliminary  testing  indicated  that  the  limited  knowledge  that  these  participants 
possessed  of  emergency  management  prevented  the  participants  from  fully  engaging  in  the 
scenario.  Roles  assigned  to  these  participants  included: 
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•  Head  of  the  Red  Cross  (National) 

•  Gapville  Fire  Department  Chief  (Local) 

•  Colonel  of  the  Reframe  Military  (Federal) 

•  Mayor  of  Gapville  (Local) 

•  Head  of  Public  Health  Agency  (Federal) 

•  Anticipate  Regional  Transport  Authority  (Provincial) 

•  RCMP  Lead  Investigator  (Federal) 

•  Chief  of  the  Gapville  Police  (Local) 

•  Director  of  the  McLaughlin  Memorial  Hospital  (Local) 

•  Head  of  Public  Safety  (Federal) 

Recruitment  for  Full  Sessions 

Forty-eight  senior  level  decision  makers  are  targeted  for  the  four  full  sessions  scheduled  to  occur 
over  April  and  May  of  this  year.  These  senior  level  decision  makers  will  be  members  of  ICS,  non- 
ICS  and  military  organizations.  See  Table  7  for  a  summary  of  the  participants  required  to 
complete  the  PODS  experiment. 

Table  7:  Summary >  of  Participants  Required  to  Complete  the  PODS  Experiment 


Phase  of  Session 

#  Military 

#ICS 

#  Non-ICS 

#  Total 

Full  Sessions 
(3  pods  plus  mixed 
closed/  substitute  pod) 

4 

4 

4 

12 

4 

4 

4 

12 

4 

4 

4 

12 

4 

4 

4 

12 

#  Total 

16 

16 

16 

48 

Full  sessions  will  require  participants  with  a  minimum  of  one  year  experience  in  a  management 
role  in  response  to  extreme  events,  previous  experience  with  at  least  one  extreme  event,  and 
decision  making  authority  within  their  respective  organization.  Inclusion  and  exclusion  criteria 
for  full  session  recruitment  sessions  are  shown  below. 

Table  8:  Inclusion  and  Exclusion  Criteria  for  Full  Sessions 
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Inclusion  and  Exclusion  Criteria 

Full  Sessions 

Previous  experience  in  both  strategic  and  operational  decision-making  during  at  least 
one  major  event 

U 

Has  been  in  management-level  decision  making  role  for  his/her  organization  for  at  least 
12  months  (or  has  was  in  a  management-level  decision  making  role  for  his/her 
organization  for  at  least  12  months  prior  to  occupying  a  new  position  or  prior  to 
retirement) 

U 

Has  the  authority  (or  has  had  the  authority  in  the  past)  to  make  decisions  regarding 
allocation  of  resources  on  behalf  of  their  organization  with  respect  to  major  events. 

u 

Able  and  willing  to  volunteer  to  participate  in  the  study 

u 

Able  and  willing  to  participate  in  English 

u 

Outside  commuting  distance  of  the  study  site 

u 

3.8.1  Communication  Process 

In  order  to  organize  the  various  participants  required  for  completion  of  the  experiment,  steps  were 
outlined  for  the  recruitment  process  and  communication  documents  were  drafted.  A  formal 
process  was  developed  for  the  recruitment  of  participants  who  were  not  solicited  through  the 
University  of  Ottawa’s  ISPR  system.  Expert  raters  contacted  during  the  scenario  and  task 
assessment  phases  and  those  who  will  be  contacted  for  the  decision  quality  rating  phase  of  the 
project  may  be  completed  via  telephone  as  participant  availability  permits.  The  formal 
recruitment  protocol  for  the  remainder  of  the  participants  in  the  PODS  project  is  as  follows: 

•  Step  1:  Review  existing  database  of  contacts 

Existing  contacts  were  reviewed  and  participants  were  selected  that  were  likely  to  meet  the 
inclusion  criteria. 

•  Step  2:  Request  additional  contacts 

Additional  contacts  were  requested  from  experts  consulted  during  the  scenario  and  task 
assessment  processes.  Senior  officials  at  the  Canadian  Forces  were  also  contacted  via  e-mail 
in  order  to  gain  assistance  with  recruiting  military  personnel. 

•  Step  3:  Invite  participants 

Participants  were  invited  via  e-mail  to  take  part  in  the  simulations.  In  some  instances, 
participants  were  contacted  by  phone  first.  An  invitation  letter  was  drafted  both  for 
organizations  and  for  individuals  (s ce  Annex  E  &  Q.  2). 

•  Step  4:  Confirm  that  interested  participants  meet  inclusion  criteria  and  determine 
availability 

Phone  confirmation  was  used  to  find  out  if  interested  participants  met  inclusion  criteria,  and 
to  determine  participant  availability.  A  phone  script  was  used  during  the  confirmation 
process  (see  Annex  Q.3).  Follow-up  occurred  by  phone  or  e-mail  as  required. 
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•  Step  5:  E-mail  reminder 

Participants  were  sent  an  e-mail  reminder  (see  Annex  Q.4)  prior  to  the  session  dates.  A  brief 
letter  was  written  which  included  the  time,  location,  date,  directions,  parking  information 
and  contact  information.  Participants  were  asked  to  contact  the  research  team  should  they 
need  to  cancel. 

•  Step  6:  Send  a  thank  you  letter 

Thank  you  letters  were  sent  to  session  participants  within  one  week  of  the  completion  of 
each  session.  These  thank  you  letters,  sent  via  e-mail,  thanked  participants  for  their 
contribution  to  the  project  (see  Annex  Q.5  &  Q.6  for  copies  of  these  letters). 

3.9  Equipment,  facilities,  software,  data  collection  and  storage 

This  section  provides  a  summary  of  the  equipment,  facilities  and  software  choices  made  for  the  in 
vivo  simulation  exercise.  (Refer  to  Annex  R  for  a  complete  equipment  and  room  requirements 
listing.)  Preliminary  equipment,  facilities  and  software  needs  were  documented  in  the 
experimental  plan  developed  under  Task  2.  However,  as  the  hardware  and  facilities  testing  took 
place  before  and  during  the  participant  pilot  tests,  these  requirements  were  modified. 

The  section  begins  with  a  discussion  on  the  use  of  video  conferencing  software  for  the  experiment 
before  describing  the  type,  extent  and  outcomes  of  the  technical  testing  conducted  prior  to  the 
start  of  the  formal  experimental  sessions  with  senior  decision-makers.  Please  note  that  Section 

3.10  Pilot  Testing  also  contains  summary  comments  related  to  technical  and  process  issues 
encountered  during  the  participant  pilot  sessions.  These  comments  were  documented  by  assigned 
observers  for  each  of  the  pilot  sessions  and  are  not  repeated  here.  The  section  ends  with  a  brief 
description  of  the  final  hardware  requirements  and  room  configurations  for  the  experiment  and 
electronic  data  collection  and  storage  protocols. 

3.9.1  Rationale  for  use  of  conference  software 

The  in  vivo  experiment  required  a  system  for  delivering  simulated  data  simultaneously  to  a 
variable  number  of  pods.  It  also  required  an  efficient  method  of  data  capture  for  text,  video  and 
audio.  This  was  particularly  important  given  the  amount  of  data  to  be  collected  over  the  course  of 
the  experiment  and  the  multiple  levels  of  analyses  to  be  performed. 

Commercially  available  video  conferencing  software  proved  to  be  an  ideal  solution  to  meeting 
these  needs.  It  offered  a  robust  study  environment  where  the  sessions  could  be  served  locally  (i.e., 
to  designated  meeting  rooms  or  working  areas)  or  remotely,  to  participants  joining  in  from  homes 
and  other  places  of  work.  It  offered  different  propriety  systems  of  data  collection,  including  video 
and  chat  recording.  With  online  video  conferencing  software,  participants  can  see  and 
communicate  with  members  of  other  pods.  Although  body  language  and  expression  can  be  very 
subtle,  it  is  a  powerful  means  of  communication  and  understanding.  The  opportunity  to  review 
and  analyze  such  data  will  inform  the  findings  of  the  experiment.  Also,  it  proved  to  be  less 
expensive  to  run  the  sessions  through  the  internet  (be  it  locally  or  remotely)  than  it  would  be  to 
hard  line  all  the  necessary  equipment  required  to  implement  the  same  experiment  with  more 
traditional  technologies. 
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Implementing  an  online  video  conferencing  solution  made  it  possible  to  create  a  functioning 
Hydra  inspired  system  of  disseminating  and  collecting  data  within  a  reasonably  limited  budget.  It 
was  also  hoped  that  by  using  online  software  the  experiment  could  be  run  remotely,  if  necessary. 


3.9.2  Software  selection  process 

A  number  of  online  video  conferencing  software  systems  were  considered  for  the  experiment, 
including  NEFSIS,  GoToMeeting,  NetMeeting,  Adobe  Connect,  DimDim  and  WebEx.  While 
there  was  overlap  in  terms  of  benefits  and  features  for  each  of  the  systems  reviewed,  the  NEFSIS 
system  came  closest  to  meeting  our  overall  requirements.  These  benefits  and  features  included: 

•  Ease  of  use  for  participants 

It  was  important  that  participants  be  able  to  easily  and  comfortably  communicate  between 
pods.  NEFSIS  offered  both  an  intuitive  and  customisable  interface. 

•  Ability  to  record  meeting  sessions 

Of  the  platforms  explored,  NEFSIS  was  the  only  one  offering  video  recording  of  meetings 
as  a  function.  With  this  function,  sessions  can  be  recorded  with  full  audio  and  video.  This 
was  an  important  factor  in  choosing  a  meeting  platform,  as  it  facilitated  the  data  collection 
process  for  multiple  media  types. 

•  Ability  to  capture  chat  messaging 

Most  of  the  platforms  reviewed  offered  methods  of  saving  chat  messages.  However, 
NEFSIS  and  Goto  Meeting  were  the  only  two  that  offered  simple  one  click  solutions. 
NEFSIS  provides  a  chat  window  allowing  participants  to  communicate  in  a  free  bi¬ 
directional  manner.  It  enables  the  saving  of  all  chat  files  as  '.txt'  files.  This  means  that  all 
chat  messaging  done  during  the  sessions  can  be  captured  for  later  review  and  content 
analysis. 

•  Security 

Due  to  ethical  considerations,  meetings  held  as  part  of  the  experiment  had  to  be  held 
securely.  Additionally,  certain  materials  needed  to  be  held  back  from  participant  until  it  was 
time  to  deliver  them  as  scenario  injects.  These  materials  remained  hidden  until  they  were 
delivered  according  to  a  pre-determined  schedule.  NEFSIS  facilitated  these  needs  through 
strict  user  access  controls.  Administrator  privileges  can  only  be  given  through  a  confidential 
login  or  directly  from  another  administrator,  meaning  that  participants  would  be  unable  to 
access  any  information  they  are  not  given.  Additionally,  these  system  controls  ensure  that 
the  system  remains  secure  from  individuals  who  are  not  invited  to  participate  in  the 
meetings. 

•  Ability  to  deliver  information  in  multiple  forms 

NEFSIS  allows  for  multiple  forms  of  information  sharing  and  exchanging.  Among  the 
options  are  screen,  application  and  multimedia  file  sharing.  For  the  purposes  of  the  in  vivo 
experiment,  NEFSIS  injects  were  delivered  exclusively  via  PowerPoint  or  multimedia 
video.  The  simulation  included  simulated  video  news  reports  as  well  as  instructions 
delivered  with  on  screen  PowerPoint  slides. 
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•  Ability  to  deliver  smooth  audio/video  files 

The  use  of  different  types  of  media  during  the  simulation  created  the  potential  for  long 
pauses  and  poor  transitions  from  one  simulation  segment  to  another.  To  limit  visual  and 
audio  distractions  and  keep  participants  engaged,  videos  needed  to  be  smooth  and  free  of 
pauses  and  glitches.  NEFSIS  was  able  to  accommodate  this  requirement. 

3.9.3  Technical  testing  of  equipment,  facilities,  and  simulation  delivery 
system 

3.9.3. 1  Technical  testing  and  equipment  and  facilities  re-configuration 

Technical  tests  were  done  on  specific  equipment  and  software  throughout  the  fall,  prior  to  the 
first  pilot  test  with  participants.  The  purpose  of  these  tests  was  to  ensure  the  proper  functioning 
and  set-up  of  the  individual  equipment  /  software  components  of  the  simulation  system.  In 
addition  to  these  technical  tests,  three  technical  run-throughs  of  the  experiment  were  conducted 
under  “full  load”  conditions.  For  these  tests,  all  required  hardware  and  rooms  were  set-up  and  the 
complete  simulation  delivered  by  video  conferencing  software.  Several  of  the  tests  involved 
practice  delivery  and  reception  of  the  simulation  from  remote  locations. 

The  nine  pilot  tests  with  participants  also  offered  additional  opportunities  to  debug 
hardware/software  issues  and  modify  equipment  settings  and  location  for  optimal  performance. 
Most  of  the  pilot  tests  were  conducted  under  “steady-state’  conditions,  i.e.,  after  all  equipment 
and  location  choices  were  finalized,  as  well  as  simulation  content,  sequence  and  processes. 

The  initial  concept  for  the  simulation  exercise  was  based  on  the  best  practices  of  several  different 
systems  used  across  Canada,  and  worldwide  in  operational  training  exercises.  The  simulation 
system  developed  at  GAP-Sante,  now  known  as  the  Inter-GAP  Simulation  System  was  intended 
for  deployment  at  an  external  off-campus  location  with  potential  delivery  to  various  selected  sites 
across  the  country.  It  was  anticipated  that  this  would  help  facilitate  recruitment  of  senior  decision 
makers  who  are  geographically  dispersed  across  Canada.  It  would  increase  the  availability  of 
participants  and  reduce  travel  time  and  associated  expenses.  In  order  to  create  this  set  up,  the 
Inter-GAP  System  required  18  laptops.  Each  participant  in  a  session  was  to  have  their  own  laptop, 
pre-loaded,  for  a  possible  12  computers  needed  for  participants.  In  addition  two  laptops  are 
needed  for  the  controllers  (one  for  open  pod  delivery  and  one  for  closed  pod  delivery)  and  four 
laptops  are  needed  for  observers.  Routers  and  switchers  were  also  obtained  for  splitting  internet 
connections  at  an  unknown  or  remote  location.  Printers  were  also  purchased  for  document  sharing 
or  inject  delivery. 

Upon  initial  testing  it  became  apparent  that  wireless  internet  connections  would  not  have 
sufficient  bandwidth  to  deliver  the  media  rich  injects  provided  in  the  scenario.  It  also  became 
apparent  that  a  technician  would  need  to  be  on  stand-by  at  any  remote  location  in  order  to  help 
the  user  download  the  software,  adjust  their  speakers  and  microphones,  and  ensure  they  were  on 
camera,  and  able  to  hear  and  see  properly.  This  negated  the  feasibility  of  having  pods  in  remote 
locations. 

It  was  decided  then  to  locate  the  experiment  entirely  at  University  of  Ottawa  facilities.  During  the 
technical  run-throughs  for  a  single  pod  using  University  of  Ottawa  student  volunteers,  it  was 


DRDC  CSS  CR  201 1-32 


39 


noted  that  immediately  after  a  simulation  inject  was  delivered,  participants  moved  away  from 
their  computer  screens  (and  web  cams)  to  sit  at  a  table  together.  In  an  attempt  to  accommodate 
this  demonstrated  behaviour,  and  to  have  a  more  realistic  experience  for  the  participant,  each 
room  was  re-configured  with  only  one  computer  and  a  large  monitor.  All  three  participants  in  a 
pod  now  shared  the  same  work  table  and  viewed  the  same  monitor.  The  use  of  a  single  computer 
per  room  was  also  supported  by  the  fact  that  most  meeting  rooms  are  configured  at  the  University 
for  only  one  internet  connection.  The  set-up  of  multiple  connections  per  room  posed  logistical 
and  security  implementation  problems.  With  all  the  pods  in  close  proximity  to  one  another,  the 
research  team  also  decided  to  introduce  the  use  of  two-way  radios  (one  per  room)  as  an 
alternative  communication  vehicle  between  pods  to  add  realism  to  the  simulation. 

3. 9. 3. 2  Technical  testing  of  simulation  system  -  summary  of  solutions 

As  alluded  to  above,  the  overall  configuration  of  the  pods  evolved  over  time  as  the  simulation 
system  was  tested  in  different  circumstances  and  environments.  In  testing  the  simulation  system, 
there  were  two  main  technical  issues  to  refine.  They  were: 

Audio/video  quality: 

Each  meeting  room  required  clear  audio  and  video  communication  between  the  pods  and  with  the 
control  room.  Without  clear  audio,  pods  would  not  be  able  to  work  together  easily,  and  without 
clear  audio  to  the  control  room,  data  recordings  could  not  be  reviewed  easily  for  analysis.  In  early 
technical  run-throughs,  it  was  determined  that  while  the  video  quality  of  the  cameras  was 
sufficient,  the  audio  was  less  than  satisfactory.  In  these  early  sessions  the  audio  microphone  was 
integrated  into  the  web  camera.  Unfortunately,  these  camera  microphones  created  significant 
feedback  issues  when  used  in  conjunction  with  the  video  camera,  computer  speakers  and  MP3 
voice  recorders.  This  issue  was  later  solved  when  USB  conferencing  speaker/microphones  were 
utilised.  The  new  microphone/speakers  featured  integrated  technology  to  cancel  out  feedback 
created  from  other  sources. 

Internet  connection  quality: 

PODS  sessions  were  run  with  both  the  control  room  and  pods  on  and  off  campus.  The  system  was 
also  tested  to  compare  its  wireless  performance  to  hard-wired  performance. 

When  the  experiment  was  run  with  computers  on  a  wireless  connection,  there  was  a  noticeable 
lag  in  communication  between  pods  and  delivery  of  injects  from  the  control  room.  Some 
participants  also  lost  their  wireless  connections  and  were  occasionally  ejected  from  the  session. 

The  session  configuration  was  also  tested  remotely  using  traditional  hard  line  internet  connections 
from  various  locations  throughout  Ottawa.  Meetings  run  in  this  fashion  worked,  however  video 
injects  sent  via  the  control  room  buffered  more  slowly  and  their  display  was  considered  to  look 
somewhat  choppy. 
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3.9.4  Equipment  and  facilities  requirements  and  set-up 


The  following  section  reflects  the  specific  equipment  and  set-up  required  to  run  the  Inter-GAP 
Simulation  System.  For  an  in  depth  description  of  the  required  equipment  and  set-up,  see  Annex  R. 

Control  Room 

The  control  room  houses  two  computers  each  with  a  dedicated  internet  connection.  One  computer 
is  used  for  running  the  open  pods  and  one  computer  is  used  for  running  the  closed  pods.  Each 
computer  must  be  connected  to  a  different  NEFSIS  meeting  room  and  have  a 
microphone/earphone  headset.  These  computers  both  need  to  be  pre-loaded  with  the  videos  and 
power  point  presentation  needed  to  complete  the  sessions.  During  each  session  the  meeting  video 
and  bi-directional  chat  is  recorded,  collected,  and  later  stored  in  a  secure  location.  Extra  seating  is 
also  available  for  observers. 

PODS  Rooms 

Each  pod  requires  a  hard-lined  internet  connection.  Rooms  must  also  be  sufficiently  large  enough 
to  contain  all  the  pod  equipment  and  the  three  participants.  Additionally,  there  must  be  sufficient 
room  such  that  a  video  camera  can  be  placed  far  enough  away  from  the  participants  to  capture  all 
of  them  in  the  camera  frame.  See  Figure  8  for  a  typical  pods  configuration. 


Figure  8:  A  typical  POD  room  configuration. 


Open  pods  are  set  up  with  one  computer  in  each  room.  A  web  camera,  and  a  microphone/speaker 
speaker  unit  are  connected  to  each  of  these  computers.  The  computers  are  connected  to  the 
NEFSIS  meeting  room  prior  to  the  start  of  the  session.  Additionally,  a  video  camera  is  set  up 
behind  each  computer  to  capture  and  record  all  the  events  within  the  room.  These  rooms  are  also 
equipped  with  a  two-way  radio  and  a  voice  recorder. 

Closed  pods  are  isolated  from  the  other  pods.  Consequently,  they  do  not  require  a  microphone  or 
web  cam.  Instead  closed  pods  are  set  up  with  one  computer,  one  voice  recorder,  and  one  video 
camera  only. 
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Brief/Debrief  Rooms 

Brief  and  debrief  rooms  need  to  be  large  enough  to  comfortably  fit  up  to  18  people.  These  rooms 
should  also  have  live  power  outlets  to  run  a  computer  and  multimedia  projector. 

3.9.5  NEFSIS  set-up  and  administration 

NEFS1S  meetings  were  initiated  by  logging  on  to  the  appropriate  meeting  through  an  assigned 
web  address.  After  entering  the  meeting  room,  administrators  adjusted  the  privileges  for  each 
pod,  such  that  participants  were  not  able  take  over  use  of  the  screen,  nor  were  participants  able  to 
present  materials.  The  administrator  also  configured  the  audio  and  video  settings,  and  pre-loaded 
all  scenario  inject  materials.  Finally,  administrators  initiated  the  recording  function  prior  to  the 
start  of  each  meeting. 

3.9.6  Data  collection  and  storage 

After  the  completion  of  each  PODS  session,  all  of  the  digital  recording  files  were  collected.  These 
files  included:  meeting  chat  files,  NEFSIS  video  recordings,  video  camera  recordings  and  digital 
voice  recordings.  All  files  were  then  uploaded  to  an  encrypted  external  hard  drive.  Materials  were 
named  using  the  session  date,  media  type,  and  pod  number.  A  backup  of  all  materials  was  also 
made  on  a  second  external  hard  drive,  which  was  also  stored  in  secure  location.  During  the 
analysis  phase  of  the  project,  materials  will  be  uploaded  to  a  secure  password  protected  folder  on 
the  network  drive  for  the  research  unit. 


3.10  Pilot  testing 

3.10.1  Overview  of  the  pilot  testing 

Pilot  testing  consisted  of  having  various  senior  managers’  help  with  the  selection  and  rating  of 
potential  scenarios.  This  was  followed  by  internal  pilot  testing,  pilot  testing  with  naive 
participants,  and  pilot  testing  with  professional  student/early  career  participants.  Pilot  testing 
concluded  with  a  senior  level  demonstration  and  review  (see  Figure  9).  Pilot  testing  consisted  of 
ten  sessions,  which  began  in  November  of  2010  and  was  completed  in  March  of  2011.  These 
sessions  varied  by  type  (coordination  and  collaboration,  open  and  closed),  number  of  participants 
and  number  of  pods  (see  Table  9). 


42 


DRDC  CSS  CR  2011-32 


Figure  9:  Review  and  Pretesting  Stages 
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Table  9:  Overview  of  the  Pilot  Tests 


Session  # 

Participant  Group 

Date 

Type 

Number  of 
participants 

Number 

of 

PODS 

Session  01 

Uottawa  students 

Tues-Nov  30 

Collaboration 

Closed 

3 

1 

Session  02 

Uottawa  students 

Wed-Dec  1 

Collaboration 

Open 

9 

3 

Session  03 

Uottawa  students 

Thurs-Dec  2 

Collaboration 

Open 

9 

3 

Session  04 

Uottawa  students 

Fri  -  Jan  21 

Coordination 

Open 

4 

2 

Session  05 

Uottawa  students 

Mon  -  Jan  24 

Collaboration 

Open 

5 

2 

Session  06 

Professional  program 

Fri-  Jan  28 

Coordination 

Open 

5 

2 

Session  07 

Professional  program 

Thurs-Feb  3 

Collaboration 

Open 

8 

3 

Session  08 

Professional  program 

Thurs-  Feb  24 

Coordination 

Open 

10 

3 

Session  09 

Professional  program 

Fri-  Feb  25 

Collaboration 

Open 

10 

3 

Session  10 

Senior  officials  "demo" 

Fri  -  March  18th 

Collaboration 

Open 

TBD 

TBD 

3.10.2  Pilot  testing  results 

The  following  section  presents  the  preliminary  results  of  the  pilot  testing  phase  of  the  PODS 
project.  Topics  addressed  include  the  content  and  process  results  of  the  pre-testing  and  pilot 
testing  phases,  the  results  from  the  senior  demonstration  session,  and  the  preliminary  quantitative 
results  of  the  selected  pilot  testing  sessions. 

3.10.2.1  Content  and  process 

Pilot  testing  sessions  yielded  important  information  concerning  required  content  and  process 
changes  in  the  in-vivo  simulation  experiment. 

Content 

The  PODS  project  simulation  materials  were  based  upon  training  exercises  used  before  the 
Vancouver  Winter  Olympic  Games.  Those  materials  were  delivered  in  the  form  of  a  live  action 
news  broadcast,  with  field  reporters.  Using  the  radiological  event  scenario,  the  simulation 
materials  in  the  PODS  project  uses  digital  avatars  to  deliver  news  broadcasts  with  field  reports. 
Due  to  the  stringent  selection,  and  intensive  subject-matter  expert  reviews  during  the  script¬ 
writing  phase  of  the  scenario,  there  were  very  few  changes  to  be  made  to  the  final  media  output. 
Feedback  from  participants  in  the  technical  run-throughs  and  pilot  sessions  were  universally 
positive  with  regards  to  the  media  delivery.  The  most  significant  change  made  was  the  addition  of 
a  new  section,  presented  as  a  "live  video  conference"  from  the  Emergency  Operations  Centre  in 
Gapville ,  delivered  by  the  Director  of  Emergency  Management.  This  device  allowed  for  further 
dissemination  of  pertinent  information.  To  better  accommodate  the  time  restraints  of  the  session, 
a  second  edit  was  made  of  the  materials,  and  the  final  output  was  shortened  by  approximately  2 
minutes.  The  paper-based  materials  in  the  participant  binders  reflected  the  changes  made  to  the 
scripts.  A  situation  report  in  a  government  standard  template,  listing  the  major  plot  points  was 
provided  for  ease  of  reference. 
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Process 


The  original  PODS  project  process  began  with  participants  filling  out  various  forms  including  the 
consent  form.  These  were  immediately  collected  by  the  team  members.  A  morning  briefing 
followed  which  introduces  participants  to  the  in-vivo  experiment.  After  the  briefing,  participants 
were  taken  to  their  respective  rooms  (or  pods)  by  team  members;  in  these  rooms  participants  were 
given  their  binders  which  contain  envelopes  with  specific  tasks  and  questionnaires.  Participants 
watched  the  simulation  and  were  prompted  visually  to  open  different  envelopes  to  discover 
informational  materials,  tasks,  and  questionnaires  for  task  one  and  task  two.  A  20  minute  break 
was  provided  to  participants  between  the  two  tasks.  Once  task  two  was  completed,  participants 
gathered  in  the  original  briefing  room  and  were  debriefed.  Participants’  feedback  was  recorded 
and  considered  along  with  team  observer  reports  on  the  technical  and  process  issues  form  (see 
Annex  P.4). 

Major  changes  to  the  original  process  included: 

•  Collecting  participant  consent  forms  and  other  initial  forms  once  participants  have  been 
brought  to  their  respective  rooms. 

•  The  addition  of  labeling  to  participant  materials  to  ensure  ethical  standards  of  confidentiality 
and  ensure  proper  linking  of  information. 

•  Inter-pod  communication  methods  were  changed,  so  that  participants  are  only  able  to 
communicate  through  CB  radios  or  the  chat  function  for  the  first  20  minutes.  Once  the  20 
minutes  are  over,  participants  are  able  to  communicate  via  video  conference  for  the 
remainder  of  the  task. 

•  Screen  layout  changes  were  made  to  the  technical  process,  and  made  a  difference 
concerning  how  videos  and  information  slides  were  to  be  presented. 

The  number  of  process  and  content  problems  decreased  considerably  from  one  session  to  the 
next.  Technical  and  process  improvements  were  incorporated  as  opportunities  arose.  The  end 
result  was  an  engaging,  efficiently  delivered  simulation  exercise  that  reflected  a  high  professional 
standard. 
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4  Next  steps:  Running  the  In  Vivo  simulation 


4.1.1  Scheduled  sessions 

A  total  of  four  additional  sessions  will  be  completed  with  senior  officials.  These  sessions  will  be 
held  on  April  6th  2011,  April  13th  2011,  May  5th  2011,  and  May  1 1th  201 1. Two  of  these  sessions 
will  have  coordinative  tasks,  while  the  other  two  will  have  collaborative  tasks.  Also,  two  of  the 
sessions  will  have  mixed  pods,  while  the  other  two  sessions  will  have  homogenous  pods.  The  mix 
of  pods  and  type  of  tasks  has  yet  to  be  assigned  to  the  specific  dates. 

4.1.2  Data  collection  methods  and  planned  analysis 

4.1. 2.1  Component  1  -  Qualitative  interviews  with  decision  makers 

Data  collection 

Additional  complementary  interviews  will  be  scheduled  according  to  the  participants’ 
availability.  They  will  also  be  audio  recorded,  and  will  take  place  in  a  convenient  location  for  the 
participant  (e.g.  her  /his  office).  These  interviews  will  take  place  in  the  National  Capital  region. 
Once  the  interviews  are  completed,  the  audio  files  will  be  secured  at  the  Gap  Sante  facilities  at 
the  University  of  Ottawa. 

Data  Transcription 

Each  interview  will  have  to  be  transcribed  completely  in  order  to  process  the  information  with 
qualitative  analysis  software  that  will  classify,  sort  out  and  range  the  data  previously  imported,  to 
find  trends  and  patterns  that  can  be  analysed. 

Data  Analysis 

The  first  step  to  analyse  the  information  transcribed,  will  be  to  detect  elements  related  to 
situational  complexity  (simple,  complicated  or  complex).  Each  section  will  contain  key  pieces  of 
data.  These  pieces  will  become  the  themes  to  develop  the  analysis  to  validate  the  inter- 
organizational  problem  solving  model  and  to  identify  themes  that  could  potentially  inform  the 
framework. 

In  order  to  accomplish  this,  data  will  be  categorized  into  few  topics.  Then  each  category  will  be 
expanded  depending  on  the  level  of  specificity  required  to  sustain  each  analysis.  The  reliability  of 
the  proposed  coding  scheme  will  be  assessed  by  comparing  the  1 0%  of  the  coding  results  of  the 
interviews,  by  two  different  researchers.  The  goal  is  to  reach  a  consensus  in  the  results,  but  if  this 
does  not  occur,  iteration  in  the  process  for  the  elements  under  discussion  is  done,  until  the  results 
concur. 
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4.1 .2.2  Component  2  -  In  vivo  simulation  experiment 

Data  Collection 


The  data  collected  will  include  both  self-report  data  collected  from  individual  participants  after 
each  task  (See  Annex  G  for  the  participant  task  questionnaires),  and  observational  data  collected 
during  the  sessions  via  audio/video  recording.  The  experiment  will  generate  at  least  seven  and  a 
half  hours  of  audio  visual  materials  per  session,  from  approximately  two  and  a  half  hours  of  pod 
interactions  at  the  different  cameras  located  in  various  rooms  (please  refer  to  Section  3.9.4  for 
details). 

Data  Cleaning  and  set  up 

All  data  will  be  set  up  for  analysis  by  checking  its  quality  and  neatness.  Data  from  the 
participants’  questionnaires  will  be  captured  in  a  digital  statistical  database,  while  protecting  the 
anonymity  of  participants  with  the  use  of  identifiers  (please  refer  to  Section  3.2.2  for  details). 

The  data  from  the  audio  and  video  recordings  will  also  be  quality  checked  and  then  correlated 
with  the  participant’s  identifiers.  Both  the  video  and  audio  recordings  will  be  processed  by 
applying  picture  filters  to  blur  faces.  Afterwards,  both  audio  and  video  data  will  be  transcribed. 
This  will  require  a  considerable  effort  due  to  the  length  of  the  recordings  and  the  complexity  of 
the  participants’  interactions. 

Data  Analysis 

The  data  will  be  coded  and  cleaned.  Descriptive  statistics  will  be  run  for  each  dependent  variable 
to  evaluate  data  characteristics  such  as  outliers,  normality,  linearity,  variance  homogeneity  and 
frequencies. 

Due  to  the  study’s  experimental  design,  it  is  expected  to  apply  variance  and  covariance  analyses 
such  as  ANOVA,  ANCOVA,  MANOVA,  and  MANCOVA.  These  analyses  will  allow  testing  the 
main  effects  of  the  two  independent  variables  (approach  to  multi-organizational  decision-making 
and  multi-organizational  environments),  and  their  interactions.  Based  on  the  results  from  the 
variance  and  covariance  analysis,  further  comparison  and  contrast  tests  would  be  required  to 
allow  for  complementary  interpretation  of  the  main  effects  and  its  interactions.  Also  based  on 
correlation  analysis,  the  relationship  between  some  dependent  variables  can  be  further 
understood. 

4.1.3  Modelling  communication  and  decision  making  functions 

The  goal  for  Task  4,  given  the  results  of  the  laboratory  findings,  is  to  develop  a  model  of  the 
emergency  management  decision  functions  and  the  communications  functions  to  support  the 
decision  function.  The  model  of  these  functions  must  describe  limitations  and  variations  based 
upon  situational  factors  described  in  the  case  studies.  These  situational  factors  must  include 
circumstances  where  there  is  the  potential  for  failure  (e.g.,  the  failure  of  the  communications 
infrastructure  during  Hurricane  Katrina).  The  functions  must  also  be  related  to  the  SDM  and  ICS 
frameworks  as  well  as  activities  described  as  elements  of  capabilities  within  the  National  Incident 
Management  System  (NIMS). 
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Annex  A  Qualitative  Interview  Guide 


Risk  Perception  and  Risk  Management:  Multi  Organizational  Problem  Solving 
INTERVIEWER  REFERENCE  SHEET 

Points  to  go  over  with  respondent  before  the  interview  starts. 

Context: 

•  These  interviews  are  one  component  of  a  larger  study  examining  multi-organizational  problem 
solving  during  emergencies  and  extreme  events.  Other  main  component  is  an  experiment  where 
organizations  go  through  an  in  vivo  exercise  and  are  required  to  problem  solve  with  other  types  of 
organizations. 

•  The  information  collected  in  the  interviews  will  help  in  interpreting  the  findings  from  the  experiment. 

Content: 

•  Interviews  are  being  conducted  with  senior  level  decision-makers  who  participated  directly  in 
planning  and/or  responding  during  key  events. 

•  Interview  should  take  about  one  hour. 

•  We  are  interested  in  your  perceptions  of  the  challenges  and  opportunities  that  exist  when  making 
decisions  with  multiple  organizations  involved. 

•  During  the  interview,  we  will: 

o  initially  go  through  some  background  about  how  you  were  involved  in  [event] 
o  then  get  you  to  focus  in  on  one  specific  challenging  situation  where  you  were  actively 
involved  in  problem  solving  and  decision-making  in  a  multi-organization  context  that  was 
particularly  challenging,  especially  one  that  involved  multiple  sectors  such  as  the 
Canadian  Forces,  or  NGOs  like  Red  Cross  or  citizen  associations,  and  multiple 
jurisdictions  such  as  Federal,  Provincial,  and  Municipal 
o  ask  you  questions  about  how  the  group  moved  through  the  problem  solving  process 
o  get  your  thoughts  on  how  some  key  concepts  we  are  using  in  the  experiment  component 
of  the  study 


Questions  before  getting  started? 
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Participant  overview 

1 .  To  get  started,  can  you  just  briefly  confirm  for  me  the  organization  you  were  with  during  the  event 
and  aspects  like  your  title,  main  role  and  responsibilities,  etc.?  [we  want  to  also  understand  their 
training  background  both  professional  and  experience-based  -  ask  here  if  natural  segue,  or  at  the 
end  of  the  interview  once  rapport  has  been  established] 

Event  overview  and  selection  of  situation 

2.  What  are  one  or  two  situations  planning  for  the  event  or  during  the  event  that  stick  out  for  you  as 
being  particularly  challenging  -  I  need  you  to  think  of  situations  that  focused  on  problem  solving  or 
decision  making,  and  that  had  a  number  of  different  types  of  organizations  actively  involved. 
Briefly,  what  would  these  be?  [who,  what,  where,  when,  why] 

a.  Which  of  these  should  we  focus  on  for  the  remainder  of  the  interview?  [get  any  additional 
details  required] 

Problem  identification  and  definition 

3.  Thinking  about  the  situation,  what  problem  was  the  most  challenging  that  had  multiple 
organizations  involved?  Who  knew  about  the  problem  first?  How  did  the  other  organizations 
become  involved?  How  was  the  problem  defined  once  other  organizations  became  involved? 
What  were  the  main  areas  of  agreement/conflict  among  the  players  involved  at  this  early  stage? 
Which  organizations’  perspectives  were  dominant  at  this  stage?  Was  there  a  lead  organization  or 
person  at  this  stage?  How  were  they  selected? 


Solution  generation 

4.  Once  the  problem  had  been  defined,  how  did  the  group  generate  various  potential  solutions?  What 
were  the  solutions  considered?  What  were  the  main  areas  of  agreement/conflict  among  the 
players  when  it  came  to  solution  generation?  Which  organizations'  perspectives  were  dominant  at 
this  stage?  Was  there  a  lead  organization  or  person  at  this  stage?  How  were  they  selected? 

Decision  making  and  implementation 

5.  Once  various  potential  solutions  had  been  generated,  how  did  the  group  decide  which  solutions  to 
implement?  What  were  the  main  areas  of  agreement/conflict  among  the  players  when  it  came  to 
deciding  on  which  solutions  to  implement?  Which  organizations’  perspectives  were  dominant  at 
this  stage?  Was  there  a  lead  organization  or  person  at  this  stage?  How  were  they  selected? 


Sharing  resources  and  flexibility 

6.  Thinking  back  to  this  specific  situation,  what  types  of  things  were  shared  across  organizations  in 
order  to  make  the  decision?  To  implement  the  solutions?  -  for  example,  information,  resources, 
leadership?  [what,  when,  who] 

Expectations  and  alternatives 

7.  Thinking  back  to  just  before  this  specific  situation  occurred,  what  would  have  been  your 
expectations  with  respect  to  how  the  situation  would  have  played  out  with  the  organizations 
involved?  Considering  how  it  rolled  out,  what  aspects  surprised  you? 
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8.  Reflecting  back  on  the  situation,  what  should  have  been  done  differently?  What  should  other 
organizations  have  done  differently?  What  would/could  your  organization  have  done  differently? 

Concepts 

9.  The  experimental  component  focuses  on  the  concepts  of  coordination,  cooperation  and 
collaboration.  When  these  terms  are  used,  what  examples  or  meanings  would  you  assign  to  each 
of  them?  Do  you  see  them  differently?  How  would  you  differentiate  them?  How  do  you  see  them 
related  to  decision-making  and  problem-solving? 
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Annex  B  Interview  Invitation  Letter 


University  d ‘Ottawa  •  University  of  Ottawa 


Email  request  to  organizations  for  recruitment 


Dear  Sir  or  Madam, 

Your  organization  is  invited  by  Dr  Louise  Lemyre  from  the  Institute  of  Population  Health  at  the  University 
of  Ottawa  to  participate  in  an  interview  on  Risk  Perception  and  Risk  Acceptability  of  Public  Crisis, 
Disasters  and  Emergencies,  by  sending  this  letter  to  some  of  your  staff  to  contact  us. 

The  goal  is  to  conduct  a  series  of  interviews  to  better  understand  key  actor’s  and  decision-maker’s 
perception  of  disasters  and  public  emergencies,  the  strategies  employed  to  solve  the  problems  faced  during 
complex  crises,  and  their  experience  working  with  representatives  from  other  organizations  to  solve  public 
emergencies.  This  part  of  the  project  is  Rinded  by  the  Public  Safety  Canada,  National  Defence  Canada,  the 
Natural  Sciences  and  Engineering  Research  Council  of  Canada,  and  the  Social  Sciences  and  Humanities 
Research  Council. 


The  participation  of  your  organization  is  important  to  us.  Managers  or  directors  from  your 
organization,  who  have  played  a  key  role  during  a  public  crisis,  such  as  the  H1N1  plan,  the  SARS  outbreak, 
the  2003  Blackout,  the  1998  Ice  storm  or  the  1997  Red  River  Floods,  are  invited  to  participate  in  an 
interview  on  risks,  directed  by  a  senior  interviewer.  An  assistant  may  also  be  present  to  take  notes,  and  a 
voice  recorder  will  be  used  to  tape  the  session.  The  identity  of  the  participants  will  not  appear  on  any  of  the 
written  transcriptions.  The  analysis  of  the  results  will  look  at  common  themes  and  ideas  among  different 
organizations.  This  will  help  to  better  understand  the  important  factors  influencing  views  on  disasters  and 
public  emergencies  in  Canadian  communities.  It  will  also  serve  to  develop  and  evaluate  best-practice 
guidelines,  to  assist  in  managing  psychosocial  considerations  of  risk  perception  and  risk  acceptability. 

The  interview  will  last  approximately  1  hour  either  at  your  offices  or  at  the  University  depending  on  the 
participants’  preference.  Questions  of  the  following  type  will  be  asked  in  relation  to  their  perception  of 
crises,  disasters  or  public  emergencies:  “From  your  perspective,  what  was  the  effect  of  multiple 
organizations  participating  in  joint  problem  solving?  How  did  you  experience  uncertainty  surrounding 
these  situations?  ”  There  are  no  right  or  wrong  answers.  We  only  want  to  know  the  participant’s  experience. 


Participation  is  anonymous.  A  code  number  will  be  attributed  to  participants.  Confidentiality  and 
anonymity  will  be  maintained  by  the  research  team.  While  global  interview  results  of  the  study  will  be 
published  and  discussed,  within  the  research  team  and  at  research  conferences,  no  identifiable  individual 
response  will  be  shared.  All  of  the  data  will  be  tabulated  and  maintained  by  the  researchers  in  accordance 
with  research  confidentiality  standards.  Participation  is  completely  voluntary  and  anonymous.  Participants 
are  free  to  withdraw  at  any  time,  refuse  to  participate,  or  refuse  to  answer  certain  questions.  There  is  no 
direct  individual  or  organizational  benefit  from  answering  the  questions.  There  is  no  expected  harm  or  risk. 


Please  contact  Dr  Louise  Lemyre’s  assistant  for  participation,  either  by  email  at 
gapsantedt  uottawa.ca  or  by  phone  at  (613)  562-5800,  extension  2321,  as  soon  as  possible. 


Sincerely, 


Professor  Louise  Lemyre,  Ph.D.,  FRSC 

®  (613)562  -  5800  (1196)  ou  (2321)  Fax:  (613)562  -  5350 
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www.gapsante.uottawa.ca 


H  louise.lemyre@uottawa.ca.  gapsante@uottawa.ca 
HPavillon  Desmarais  Hall,  55  Laurier  Ave  E,  piece/room  3217 

Ottawa  (ON),  Canada,  KIN  6N5 
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Annex  C  Interview  Consent  Form 


Universite  d’Ottawa  •  University  of  Ottawa 


Consent  form  and  information  sheet 


Dear  Sir  or  Madam, 

You  are  invited  by  Dr  Louise  Lemyre  from  the  Institute  of  Population  Health  at  the  University  of  Ottawa  to 
participate  in  an  interview  on  Risk  Perception  and  Risk  Acceptability  of  Crises,  Disasters  and  Emergencies. 

The  goal  is  to  conduct  a  series  of  interviews  to  better  understand  the  key  decision  maker's  perception  of 
disasters  and  public  emergencies,  the  strategies  employed  by  them  to  solve  the  problems  faced  during  these 
complex  crises,  and  their  experience  working  with  representatives  from  other  organizations  to  solve  public 
emergencies.  This  part  of  the  project  is  funded  by  Public  Safety  Canada,  National  Defence  Canada,  the 
Natural  Sciences  and  Engineering  Research  Council  of  Canada,  and  the  Social  Sciences  and  Humanities 
Research  Council. 

Your  opinion  is  important  to  us.  You  will  be  asked  to  participate  in  an  interview  and  to  answer  questions 
directed  by  a  trained  interviewer.  An  assistant  may  also  be  present  to  take  notes  and  a  voice  recorder  will  be 
used  to  tape  the  session.  All  of  the  discussion  will  be  transcribed  and  entered  in  a  computer  fde  for  analysis 
of  content.  The  data  will  be  conserved  for  a  maximum  of  10  years  in  a  locked  cabinet  in  the  Gap-Sante 
office  at  the  University  of  Ottawa  and  will  be  shredded  at  the  end  of  storage.  Your  identity  will  not  appear 
on  any  of  the  written  transcriptions.  The  analysis  of  the  results  will  look  at  common  themes  and  ideas 
among  key  decision  makers  from  different  organizations.  This  will  help  to  better  understand  the  important 
factors  influencing  views  on  disasters  and  public  emergencies  in  Canadian  communities.  It  will  also  serve 
to  develop  and  evaluate  best-practice  guidelines,  to  assist  health  professionals  and  other  key  players  to 
effectively  manage  the  short-term,  mid-term  and  long-term  psychosocial  consequences  of  crises,  disasters 
and  emergency  events. 

The  interview  will  last  approximately  1  hour.  Questions  of  the  following  type  will  be  asked  in  relation  to 
your  perception  of  disasters  and  public  emergencies:  “From  your  perspective,  what  was  the  effect  of 
multiple  organizations  participating  in  joint  problem  solving?  Flow  did  you  experience  uncertainty 
surrounding  these  situations?  ”  Your  answers  can  be  general  and  you  do  not  have  to  reveal  any  information 
that  you  do  not  want  to.  There  are  no  right  or  wrong  answers.  We  only  want  to  know  your  experience. 

Your  participation  is  anonymous.  Do  not  use  your  full  name.  Confidentiality  and  anonymity  will  be 
maintained  by  the  research  team.  While  interview  results  of  the  study  will  be  published  and  discussed 
within  the  research  team,  and  at  research  conferences,  no  one  individual’s  responses  to  the  questions  will 
be  shared.  All  of  the  data  will  be  tabulated  and  maintained  by  the  researchers  in  accordance  with  research 
confidentiality  standards. 

Participation  is  completely  voluntary  and  anonymous.  You  are  free  to  withdraw  at  any  time,  refuse 
to  participate,  or  refuse  to  answer  certain  questions.  There  is  no  direct  individual  benefit  from 
answering  the  questions.  There  is  no  expected  harm  or  risk  except  the  possible  negative  feelings 
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sometime  associated  with  self-reflecting  on  crises,  disasters  and  public  emergencies.  However,  in  the 
unlikely  event  of  distress  or  discomfort  you  may  wish  to  contact  the  Confidential  Help  Line  within 
your  locality.  You  may  also  contact  Dr  Louise  Lemyre  at  the  University  of  Ottawa  at  613-562-5800, 
extension  1196,  or  the  University  of  Ottawa  Research  Ethics  Board  Officer  at  613-562-5387 
(ethics@uottawa.ca). 

There  are  two  copies  of  this  form,  one  for  you,  one  for  the  research  team. 


Louise  Lemyre,  Ph.D. 

University  of  Ottawa  Professor 

I  agree  to  participate  in  this  study  about  Risk  Perception  and  Risk  Acceptability  of  Crises,  Disasters 
and  Public  Emergencies  conducted  by  Dr  Louise  Lemyre. 


Participant,  date 
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Annex  D  Ethics  Approval  Letter 


Hlr  Number:  08*10-31 


Uni  vers  ite  d’ Ottawa 

llurcau  dtMhiquc  cl  d 'integrity  dc  la  nrchcrchc 


Date  (mm  (W/I4/20I0 

teg*' 

University  of  Ottawa 

( Hike  4»f  Research  I  Ibics  and  Integrity 


This  is  to  confirm  [lull  ihe  University  of  Ottawa  Research  Ethics  Board  identified  above,  which  operates  in 
accordance  with  the  Tri-Council  Policy  Statement  and  other  applicable  laws  and  regulations  in  Ontario,  has 
examined  and  approved  the  application  for  ethical  approval  tor  the  above  named  research  project  as  of  the 
Ethics  Approval  Date  indicated  for  the  period  above  and  subject  to  the  conditions  listed  the  section  above 
entitled  "Special  Conditions  /  Comments". 

During  Ihe  course  of  the  stud)  Ihe  protocol  may  not  lie  modified  w  ithout  prior  written  approval  from  Ihe  R  f  It 
except  when  necessary  to  remove  subjects  from  immediate  endangerment  or  when  the  modification!  s)  pertain 
lo  only  administrative  or  logistical  components  of  the  study  (e.g.  change  of  telephone  number).  Investigators 
must  also  promptly  alert  the  REB  of  any  changes  which  increase  ihe  risk  to  participant(s).  any  changes  which 
considerably  affect  the  conduct  of  Ihe  project,  all  unanticipated  and  harmful  events  dial  occur,  and  new 
information  that  may  negatively  altcel  the  conduct  of  the  ptoject  and  safely  of  Ihe  participantfs).  Modifications 
lo  ihe  project,  information/consent  documentation,  and/or  recruitment  documentation,  should  be  submitted  lo 
this  office  for  approval  using  the  "Modification  to  research  project”  form  available  at: 
hllp://w  w  w  .rges.uoltavva  ca/elhics/application  dw  n.asp 

Please  submit  an  alumni  status  report  lo  Ihe  Protocol  Officer  4  weeks  before  the  above-referenced  expiry  dale 

10  either  close  the  file  or  request  a  renewal  of  ethics  approval.  This  document  can  be  found  at; 
http://www.rges.uotiawa.ca/ethics/applicalion  dwn.asp 

11  you  have  any  questions,  please  do  not  hesitate  lo  contact  the  Ethics  Office  at  extension  5X4 1  or  by  e-mail  at. 
ethics® uOttawa.cn. 


Signature: 

I'Pjohfrf' 

Isabelle  Robert. 

Protocol  Officer  lor  Ethics  in  Research 
For  Barbara  Graves. 

Chair  of  the  Social  Sciences  and  Humanities  REB 


2 


550,  ate  Cumberland  550  Cumberland  Sirett 

i JltOvva (Ontario) KIN  6N5  Canada  oiuiwu.  Oniutio  KIN 6N5  Cuiuulu 
(613)  562-584 1  -  TcIcc.  I  un  (6131  562-5338 
hUp^  ,w\\\v,rcKcarch.ut>iuma.L,u  »:llncs'rjndL*vliiml 
blip” 'www  rcchcTche-uoiiawica/deoninJopic/indcvluml 


DRDC  CSS  CR  201 1-32 


57 


Annex  E  Invitation  Email  for  Recruitment 


Email  request  to  organizations  for  recruitment 


Dear  Sir  or  Madam, 

I  am  inviting  representatives  from  your  organization  to  participate  in  a  study  at  the 
University  of  Ottawa.  This  study  has  been  designed  to  help  us  better  understand  how 
people  in  decision  making  roles  perceive  risks,  assess  problems  and  manage  threats 
during  emergencies.  This  study  involves  having  people  from  different  organizations 
participate  in  a  session  that  is  similar  to  a  table-top  exercise  that  focuses  on  a  specific 
emergency  scenario.  The  title  of  the  project  is  Risk  Management  and  Governance: 
Understanding  Problem  Solving  and  Decision  Making,  and  it  is  being  funded  by  the 
Social  Sciences  and  Humanities  Research  Council  and  Defence  Research  Development 
Canada. 

Who  should  participate? 

We  are  hoping  some  representatives  from  your  organization  are  able  to  participate  in  this 
study.  The  participants  for  this  study  will  consist  of  higher-level  decision  makers  who 
have  had  some  previous  experience  in  both  strategic  and  operational  decision-making  in 
managing  and  responding  to  threats.  They  should  have  been  in  a  management-level 
decision  making  role  within  your  organization  for  at  least  one  year.  We  will  be  inviting 
participants  from  various  organizations  to  participate  in  the  same  session. 

What  does  participation  involve? 

The  session  will  last  approximately  three  hours  and  will  resemble  a  table-top  exercise 
often  used  for  emergency  planning.  The  session  will  take  place  at  the  University  of 
Ottawa  downtown  campus  during  regular  business  hours.  Participants  will  be  asked  to 
interact  with  other  session  participants  as  they  work  through  the  exercises.  As  well,  they 
will  be  asked  to  complete  various  questionnaires  on  perceptions  and  opinions  of  tasks, 
perfonnance  and  interactions.  All  sessions  will  be  video  and  audio  recorded  for  data 
collection  purposes.  All  aspects  of  participation  are  voluntary.  Participants  will  be  asked 
to  read  through  and  sign  a  consent  form  indicating  infonned  consent  prior  to  their 
participation  in  the  session. 

What  about  confidentiality  and  anonymity? 

Data  collected  from  participants  will  remain  confidential.  The  only  people  who  will  have 
access  to  identifiable  data  will  be  members  of  the  research  team.  Working  files  such  as 
transcripts  and  questionnaires  will  have  identifiers  (e.g.,  names,  organization)  removed 
and  replaced  by  an  identification  number.  The  questionnaires,  audio  and  video  tapes  and 
the  transcriptions  will  be  kept  in  a  secure  manner  in  a  laboratory  at  the  University  of 
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Ottawa  to  which  only  the  researchers  have  access  for  a  period  of  ten  years.  Participation 
in  the  study  will  not  be  completely  anonymous,  as  the  participant’s  organization  will 
know  that  he  or  she  is  participating,  as  will  the  fellow  session  participants.  Beyond  this, 
the  research  team  will  protect  the  anonymity  of  participants  by  ensuring  that  participants’ 
names  are  not  provided  to  any  group  outside  of  the  research  team  directly  involved  with 
this  study. 

What  are  the  individual  level  risks  and  benefits  associated  with  participating  in  this 
study? 

Participation  is  completely  voluntary.  Participants  are  free  to  withdraw  at  any  time,  refuse  to 
participate,  or  choose  to  not  answer  certain  questions.  We  do  not  expect  that  there  will  be  any 
harm  or  potential  risks  with  the  exception  of  potentially  negative  feelings  sometimes  associated 
with  working  directly  with  materials  that  focus  on  emergencies.  The  benefit  of  participation  is 
that  the  study  will  provide  useful  information  for  the  emergency  planning  and  response 
community  more  broadly  on  how  to  potentially  improve  decision-making  and  management  of 
emergency  events. 

What  are  the  next  steps? 

A  member  of  my  research  team  will  be  in  contact  with  you  either  by  phone  or  email  over 
the  next  few  days  to  determine  if  there  are  any  members  of  your  organization  that  would 
be  interested  in  participating  and  how  they  can  get  in  contact  with  the  research  team,  and 
to  answer  any  additional  questions  you  may  have  about  the  study.  In  the  mean  time,  if 
you  have  questions  or  would  like  further  information,  please  contact  me  directly  at  (613) 
562-5800  x  2321  or  gapsante@uottawa.ca  . 

Sincerely, 


Louise  Lemyre,  Ph.D.,  MSRC|FRSC 
Professeure  titulaire  |  Full  Professor 

Ecole  de  psychologie,  Faculte  des  sciences  sociales  |  School  of  Psychology,  Faculty  of 
Social  Sciences 

Chaire  McLaughlin  sur  le  risque  psychosocial  |  The  McLaughlin  Chair  on  Psychosocial 
Risk 

Directrice  de  l'unite  de  recherche  GAP-Sante  |  GAP-Sante  research  unit  Director 

Institut  de  sante  des  populations  |  Institute  of  Population  Health 

Universite  d’Ottawa  |  University  of  Ottawa 

DMS  3215,  55  LaurierE 

Ottawa,  On,  CANADA  KIN  6N5 

(+1)  613-562-5800  xl  196  ,  assist.  x2321 

louise.lemyre@uottawa.ca ;  gapsante@uottawa.ca 

www.gapsante.uottawa.ca 
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Annex  F  Participant  Consent  Form 


Universite  d'Otiawa  •  University  of  Ottawa 


Information  Sheet  and  Consent  form 

You  have  been  invited  to  participate  in  a  study  entitled  Risk  Management  and  Governance: 
Understanding  Problem  Solving  and  Decision  Making:  "Problem-solving  and  Organizational 
Decision-making  (PODS)  Project"  (the  "Project")This  document  provides  information  on  the 
study  including  the  overall  purpose  of  the  study,  what  is  involved  in  participating,  possible  risks 
and  benefits,  how  confidentiality  and  anonymity  will  be  protected  by  the  researchers,  how  data 
will  be  conserved,  and  the  voluntary  nature  of  the  study.  As  you  read  through  this  information 
sheet,  please  note  any  questions  that  you  have  or  anything  that  you  would  like  to  have  clarified 
by  the  researcher  before  you  sign  your  acceptance  to  participate  in  the  study. 


Risk  Management  and  Governance:  Understanding  Problem 
Solving  and  Decision  Making:  "Problem-solving  and 
Organizational  Decision-making  (PODS)  Project" 

Louise  Lemyre 

Professor,  School  of  Psychology 
Faculty  of  Social  Sciences 
University  of  Ottawa 
Tel:  613-562-5800  (ext  1196) 
llemyre(5)uottawa.ca 

Funders:  This  study  is  funded  by  the  Social  Sciences  and  Humanities  Research  Council,  with 
additional  funding  from  Defense  Research  Development  Canada. 

Purpose  of  the  Study:  I  understand  that  the  purpose  of  the  study  is  to  better  understand  how 
people  in  decision  making  roles  perceive  risks,  assess  problems  and  manage  threats  during 
emergencies.  This  study  involves  having  people  from  different  organizations  participate  in  a 
session  that  is  similar  to  a  table-top  exercise  that  focuses  on  a  specific  emergency  scenario. 

Participation:  My  participation  in  the  study  will  consist  of  one  session  of  approximately  four 
hours,  located  at  the  downtown  campus  of  the  University  of  Ottawa.  During  the  session  I  will  be 
asked  to  participate  in  an  exercise  that  resembles  a  table-top  exercise  often  used  for  emergency 
planning.  I  and  the  other  participants  assigned  to  my  group  for  the  session  will  be  presented 
with  a  scenario  and  asked  to  complete  problem-solving  tasks  related  to  the  scenario.  I  will  be 
asked  to  interact  with  other  session  participants  as  I  work  through  the  tasks.  As  well,  I  will  be 
asked  to  complete  questionnaires  that  ask  about  my  perceptions  and  opinions  of  the  tasks, 


Title  of  the  study: 


Principal  Investigator: 
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performance  and  interactions.  The  session  will  be  video  and  audio  recorded  for  data  collection 
purposes,  and  later  transcribed  in  an  electronic  file. 

Risks:  I  understand  that  this  research  is  devoid  of  physical  and  psychological  risks  other  than  the 
possible  mild  negative  feelings  that  could  be  associated  with  working  with  a  scenario  that 
focuses  on  an  emergency  situation.  The  research  team  has  selected  the  materials  and 
developed  the  scenario  in  a  manner  that  attempts  to  minimize  this  risk  by  not  including  graphic 
descriptions  or  negative  images. 

Benefits:  My  participation  in  this  study  will  help  improve  understanding  and  provide  useful 
information  for  the  emergency  planning  and  response  community  on  how  to  potentially 
improve  decision-making  and  management  of  emergency  events. 

Confidentiality  and  Anonymity:  I  have  received  assurance  from  the  researcher  that  the 
information  I  will  share  will  remain  strictly  confidential.  I  understand  that  the  contents  will  be 
used  only  for  research  purposes  and  that  my  confidentiality  will  be  protected:  only  grouped 
data  will  be  reported.  My  participation  in  the  study  will  not  be  completely  anonymous,  as  the 
fellow  session  participants  will  know  that  I  am  participating.  Beyond  this,  the  research  team  will 
protect  my  anonymity  by  ensuring  that  my  name  is  not  provided  to  any  group  outside  of  the 
research  team  directly  involved  with  this  study. 

Conservation  of  Data:  The  questionnaires,  audio  and  video  tapes  and  the  transcriptions  will  be 
kept  in  a  secure  manner  in  a  laboratory  at  the  University  of  Ottawa  to  which  only  the 
researchers  have  access  for  a  period  of  ten  years. 

Voluntary  Participation:  I  am  under  no  obligation  to  participate  and  if  I  choose  to  participate,  I 
may  withdraw  from  the  study  at  any  time.  I  may  also  refuse  to  answer  any  questions,  without 
suffering  any  negative  consequences.  If  I  choose  to  withdraw,  all  data  gathered  until  the  time  of 
withdrawal  will  be  destroyed  and  will  not  be  used. 

Acceptance:  Participation  is  completely  voluntary.  You  are  free  to  withdraw  at  any  time,  refuse 
to  participate,  or  refuse  to  answer  certain  questions.  There  is  no  direct  individual  benefit  from 
answering  the  questions.  There  is  no  expected  harm  or  risk  except  the  possible  negative  feelings 
sometime  associated  with  self-reflecting  on  public  emergencies.  However,  in  the  unlikely  event 
of  distress  or  discomfort  you  may  wish  to  contact  the  Confidential  Help  Line  within  your  locality. 
You  may  also  contact  Dr  Louise  Lemyre  at  the  University  of  Ottawa  at  613-562-5800,  extension 
1196,  or  the  University  of  Ottawa  Research  Ethics  Board  Officer  at  613-562-5387,  University  of 
Ottawa,  Tabaret  Hall,  550  Cumberland  Street,  Room  159,  Ottawa,  ON  (ethics@uottawa.ca).l 
agree  to  participate  in  this  study  Risk  Management  and  Governance:  Understanding  Problem 
Solving  and  Decision  Making:  "Problem-solving  and  Organizational  Decision-making  (PODS) 
Project'  conducted  by  Dr  Louise  Lemyre. 


PARTICIPANT: 


DATE: 
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Louise  Lemyre,  Ph.D. University  of  Ottawa  Professor 


GAP<Sante 
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Annex  G  Task  Questionnaires 


G.1  Open  Pod  Task  #1  Questionnaire 


Do  not  complete  this  questionnaire  until  instructed  to  do  so 


Individual  Participation  and  Perspectives  -  Task  Questionnaire 

Please  reflect  on  your  own  participation  and  thoughts  during  the  task  that  you  just  completed. 
Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a  scale  of  0  to  4 
(0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 

disagree  Disagree  Neutral  Agree  agree 
0  12  3  4 


Ql 

I  am  satisfied  with  the 

problem  solving  processes  I 
used  during  the  task. 

o 

o 

o 

o 

o 

Q2 

I  am  satisfied  with  the 

opportunities  I  had  to  provide 
input. 

o 

o 

o 

o 

o 

Q3 

I  am  satisfied  with  the  overall 
quality  of  the  outcome  from 
the  task. 

o 

o 

o 

o 

o 

Q4 

I  am  frustrated  working  with 
the  people  in  my  pod  group. 

o 

o 

o 

o 

o 

Q5 

I  am  frustrated  working  with 
the  people  in  the  other  pods. 

o 

o 

o 

o 

o 

Q6 

1  participated  actively  in  the 
decision  making  process. 

o 

o 

o 

o 

o 

Q7 

I  participated  in  a  leadership 
role  within  my  own  pod. 

o 

o 

o 

o 

o 

Q8 

I  participated  in  a  leadership 
role  across  the  other  pods. 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q9 

I  facilitated  discussion  within 
my  own  pod. 

o 

o 

o 

o 

o 

Q10 

I  facilitated  discussion 
between  pods. 

o 

o 

o 

o 

o 

Qll 

I  recorded  information  on 
behalf  of  my  own  pod. 

o 

o 

o 

o 

o 

Q12 

I  recorded  information  on 
behalf  of  all  pods 

o 

o 

o 

o 

o 

Q13 

I  was  frustrated  by 
differences  of  opinion  within 
my  pod  during  the  task. 

o 

o 

o 

o 

o 

Q14 

I  was  frustrated  by 
differences  of  opinion 
between  the  other  pods 
during  the  task. 

o 

o 

o 

o 

o 

Q15 

I  trust  the  people  in  my  pod. 

o 

o 

o 

o 

o 

Q16 

I  trust  the  people  in  the  other 
pods. 

o 

o 

o 

o 

o 

Q17 

I  felt  a  sense  of  belonging 
within  my  pod. 

o 

o 

o 

o 

o 

Q18 

I  felt  a  sense  of  belonging 
with  the  other  pods. 

o 

o 

o 

o 

o 

Q19 

I  agree  with  the  decisions  and 
outcomes  from  the  task. 

o 

o 

o 

o 

o 

Q20 

Prior  to  today’s  session  I 
would  describe  my 
relationship  with  at  least  one 
of  the  people  in  my  pod  as  a 
friendship. 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q22 

Prior  to  today’s  session,  I 
have  worked  with  at  least  one 
of  the  people  in  my  pod. 

o 

o 

o 

o 

o 

Q24 

There  was  enough  time 
allotted  to  solve  the  problems 
during  the  task. 

o 

o 

o 

o 

o 

Q25 

The  information  and 
instructions  given  by  the 
session  facilitator  were  clear. 

o 

o 

o 

o 

o 

Q26 

The  task  was  difficult  to 
complete. 

o 

o 

o 

o 

o 
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Within  POD  Participation  and  Perspectives  -  Task  Questionnaire 


Please  reflect  on  the  participation  of  the  various  people  in  your  pod  during  the  task  that  you 
just  completed.  Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a 
scale  of  0  to  4  (0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Q27 

The  outcome  of  the  task 
reflects  the  input  from  all  the 
people  in  my  pod. 

o 

o 

o 

O 

o 

Q28 

The  differences  of  opinion 
within  my  pod  helped  us 
reach  better  decisions  during 
the  task. 

o 

o 

o 

o 

o 

Q29 

The  people  in  my  pod 
actively  participated  in  the 
problem  solving  process. 

o 

o 

o 

o 

o 

Q30 

The  people  in  my  pod 
communicated  effectively 
with  one  another. 

o 

o 

o 

o 

o 

Q31 

The  people  in  my  pod  were 
engaged  in  the  decision 
making  process. 

o 

o 

o 

o 

o 

Q32 

The  people  in  my  pod 
generated  various  alternative 
ideas. 

o 

o 

o 

o 

o 

Q33 

A  clear  leader  emerged 
within  my  pod. 

o 

o 

o 

o 

o 

Q34 

There  was  more  than  one 
leader  within  my  pod. 

o 

o 

o 

o 

o 

Q35 

Personality  styles  within  my 
pod  got  in  the  way  of 
completing  the  task. 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q36 

The  people  within  my  pod 
had  frequent  differences  of 
opinion. 

o 

o 

o 

O 

o 

Q37 

The  people  in  my  pod  were 
motivated  to  complete  the 
task. 

o 

o 

o 

o 

o 

Q38 

The  people  in  my  pod  were 
able  to  stay  focused  on  the 
task. 

o 

o 

o 

o 

o 

DRDC  CSS  CR  201 1-32 


67 


Between  PODS  Participation  and  Perspectives  -  Task  Questionnaire 

Please  reflect  on  the  participation  of  people  from  all  the  pods  during  the  task  that  you  just 
completed.  Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a 
scale  of  0  to  4  (0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly 

disagree 

Disagree 

Neutral 

Agree 

Strongly 

agree 

0 

1 

2 

3 

4 

Q39 

The  differences  of  opinion 
across  the  pods  helped  us  to 
reach  a  better  decision  during 
the  task. 

o 

o 

o 

o 

o 

Q40 

People  from  the  other  pods 
actively  participated  with 
people  from  my  pod  in  the 
problem  solving  process. 

o 

o 

o 

o 

o 

Q41 

The  pods  communicated 
effectively  with  one  another. 

o 

o 

o 

o 

o 

Q42 

People  from  the  other  pods 
were  engaged  in  the  decision 
making  process. 

o 

o 

o 

o 

o 

Q43 

People  from  the  other  pods 
generated  various  alternative 
ideas. 

o 

o 

o 

o 

o 

Q44 

A  clear  leader  emerged 
between  the  pods. 

o 

o 

o 

o 

o 

Q45 

Personality  styles  between 
people  in  the  different  pods 
got  in  the  way  of  completing 
the  task. 

o 

o 

o 

o 

o 

Q46 

People  in  the  other  pods  had 
frequent  differences  of 
opinion. 

o 

o 

o 

o 

o 

Q47 

The  people  from  the  other 
pods  were  motivated  to 
complete  the  task. 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q48 

The  pods  were  able  to  stay 
focused  on  the  task. 

o 

o 

o 

O 

o 

Q49 

Decisions  made  between  my 
pod  and  the  other  pods  were 
consensus-based. 

o 

o 

o 

o 

o 

Q21 

Prior  to  today’s  session  I 
would  describe  my 
relationship  with  at  least  one 
of  the  people  in  the  other 
pods  as  a  friendship. 

o 

o 

o 

o 

o 

Q23 

Prior  to  today’s  session,  I 
have  worked  with  at  least  one 
of  the  people  from  the  other 

o 

o 

o 

o 

o 

pods. 
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G.2  Open  Pod  Task  #2  Questionnaire 


Do  not  complete  this  questionnaire  until  instructed  to  do  so 


Individual  Participation  and  Perspectives  -  Task  Questionnaire 

Please  reflect  on  your  own  participation  and  thoughts  during  the  task  that  you  just  completed. 
Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a  scale  of  0  to  4 
(0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Ql 

I  am  satisfied  with  the 
problem  solving  processes  I 
used  during  the  task. 

o 

o 

o 

O 

o 

Q2 

I  am  satisfied  with  the 
opportunities  I  had  to  provide 
input. 

o 

o 

o 

o 

o 

Q3 

I  am  satisfied  with  the  overall 
quality  of  the  outcome  from 
the  task. 

o 

o 

o 

o 

o 

Q4 

I  am  frustrated  working  with 
the  people  in  my  pod  group. 

o 

o 

o 

o 

o 

Q5 

I  am  frustrated  working  with 
the  people  in  the  other  pods. 

o 

o 

o 

o 

o 

Q6 

1  participated  actively  in  the 
decision  making  process. 

o 

o 

o 

o 

o 

Q7 

1  participated  in  a  leadership 
role  within  my  own  pod. 

o 

o 

o 

o 

o 

Q8 

1  participated  in  a  leadership 
role  across  the  other  pods. 

o 

o 

o 

o 

o 

Q9 

I  facilitated  discussion  within 
my  own  pod. 

o 

o 

o 

o 

o 
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Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Q10 

I  facilitated  discussion 
between  pods. 

o 

o 

o 

O 

o 

Qll 

I  recorded  information  on 
behalf  of  my  own  pod. 

o 

o 

o 

o 

o 

Q12 

I  recorded  information  on 
behalf  of  all  pods 

o 

o 

o 

o 

o 

Q13 

I  was  frustrated  by 
differences  of  opinion  within 
my  pod  durins  the  task. 

o 

o 

o 

o 

o 

Q14 

I  was  frustrated  by 
differences  of  opinion 
between  the  other  pods 
during  the  task. 

o 

o 

o 

o 

o 

Q15 

I  trust  the  people  in  my  pod. 

o 

o 

o 

o 

o 

Q16 

I  trust  the  people  in  the  other 
pods. 

o 

o 

o 

o 

o 

Q17 

I  felt  a  sense  of  belonging 
within  my  pod. 

o 

o 

o 

o 

o 

Q18 

I  felt  a  sense  of  belonging 
with  the  other  pods. 

o 

o 

o 

o 

o 

Q19 

I  agree  with  the  decisions  and 
outcomes  from  the  task. 

o 

o 

o 

o 

o 

Q24 

There  was  enough  time 
allotted  to  solve  the  problems 
during  the  task. 

o 

o 

o 

o 

o 

Q25 

The  information  and 
instructions  given  by  the 
session  facilitator  were  clear. 

o 

o 

o 

o 

o 

Q26 

The  task  was  difficult  to 
complete. 

o 

o 

o 

o 

o 
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Within  POD  Participation  and  Perspectives  -  Task  Questionnaire 


Please  reflect  on  the  participation  of  the  various  people  in  your  pod  during  the  task  that  you 
just  completed.  Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a 
scale  of  0  to  4  (0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Q27 

The  outcome  of  the  task 
reflects  the  input  from  all  the 
people  in  my  pod. 

o 

o 

o 

O 

o 

Q28 

The  differences  of  opinion 
within  my  pod  helped  us 
reach  better  decisions  during 
the  task. 

o 

o 

o 

o 

o 

Q29 

The  people  in  my  pod 
actively  participated  in  the 
problem  solving  process. 

o 

o 

o 

o 

o 

Q30 

The  people  in  my  pod 
communicated  effectively 
with  one  another. 

o 

o 

o 

o 

o 

Q31 

The  people  in  my  pod  were 
engaged  in  the  decision 
making  process. 

o 

o 

o 

o 

o 

Q32 

The  people  in  my  pod 
generated  various  alternative 
ideas. 

o 

o 

o 

o 

o 

Q33 

A  clear  leader  emerged 
within  my  pod. 

o 

o 

o 

o 

o 

Q34 

There  was  more  than  one 
leader  within  my  pod. 

o 

o 

o 

o 

o 

Q35 

Personality  styles  within  my 
pod  got  in  the  way  of 
completing  the  task. 

o 

o 

o 

o 

o 

Q36 

The  people  within  my  pod 
had  frequent  differences  of 
opinion. 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q37 

The  people  in  my  pod  were 
motivated  to  complete  the 
task. 

o 

o 

o 

O 

o 

Q38 

The  people  in  my  pod  were 
able  to  stay  focused  on  the 
task. 

o 

o 

o 

o 

o 
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Between  PODS  Participation  and  Perspectives  -  Task  Questionnaire 


Please  reflect  on  the  participation  of  people  from  all  the  pods  during  the  task  that  you  just 
completed.  Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a 
scale  of  0  to  4  (0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly 

disagree 

Disagree 

Neutral 

Agree 

Strongly 

agree 

0 

1 

2 

3 

4 

Q39 

The  differences  of  opinion 
across  the  pods  helped  us  to 
reach  a  better  decision  during 
the  task. 

o 

o 

o 

o 

o 

Q40 

People  from  the  other  pods 
actively  participated  with 
people  from  my  pod  in  the 
problem  solving  process. 

o 

o 

o 

o 

o 

Q41 

The  pods  communicated 
effectively  with  one  another. 

o 

o 

o 

o 

o 

Q42 

People  from  the  other  pods 
were  engaged  in  the  decision 
making  process. 

o 

o 

o 

o 

o 

Q43 

People  from  the  other  pods 
generated  various  alternative 
ideas. 

o 

o 

o 

o 

o 

Q44 

A  clear  leader  emerged 
between  the  pods. 

o 

o 

o 

o 

o 

Q45 

Personality  styles  between 
people  in  the  different  pods 
got  in  the  way  of  completing 
the  task. 

o 

o 

o 

o 

o 

Q46 

People  in  the  other  pods  had 
frequent  differences  of 

o 

o 

o 

o 

o 

opinion. 

Q47 

The  people  from  the  other 
pods  were  motivated  to 
complete  the  task. 

o 

o 

o 

o 

o 
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Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Q48 

The  pods  were  able  to  stay 
focused  on  the  task. 

o 

o 

o 

O 

o 

Q49 

Decisions  made  between  my 
pod  and  the  other  pods  were 

o 

o 

o 

O 

o 

consensus-based. 


DRDC  CSS  CR  201 1-32 


75 


G.3  Closed  Pod  Task  #1  Questionnaire 


Do  not  complete  this  questionnaire  until  instructed  to  do  so 


Individual  Participation  and  Perspectives  -  Task  Questionnaire 

Please  reflect  on  your  own  participation  and  thoughts  during  the  task  that  you  just  completed. 
Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a  scale  of  0  to  4 
(0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Ql 

I  am  satisfied  with  the 
problem  solving  processes  I 
used  during  the  task. 

o 

o 

o 

O 

o 

Q2 

I  am  satisfied  with  the 
opportunities  I  had  to  provide 
input. 

o 

o 

o 

o 

o 

Q3 

I  am  satisfied  with  the  overall 
quality  of  the  outcome  from 
the  task. 

o 

o 

o 

o 

o 

Q4 

I  am  frustrated  working  with 
the  people  in  my  pod  group. 

o 

o 

o 

o 

o 

Q6 

1  participated  actively  in  the 
decision  making  process. 

o 

o 

o 

o 

o 

Q7 

1  participated  in  a  leadership 
role  within  my  own  pod. 

o 

o 

o 

o 

o 

Q9 

I  facilitated  discussion  within 
my  own  pod. 

o 

o 

o 

o 

o 

Qll 

I  recorded  information  on 
behalf  of  my  own  pod. 

o 

o 

o 

o 

o 

Q13 

I  was  frustrated  by 
differences  of  opinion  within 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

my  pod  during  the  task. 

Q15 

I  trust  the  people  in  my  pod. 

o 

o 

o 

O 

o 

Q17 

I  felt  a  sense  of  belonging 
within  my  pod. 

o 

o 

o 

o 

o 

Q19 

I  agree  with  the  decisions  and 
outcomes  from  the  task. 

o 

o 

o 

o 

o 

Q20 

Prior  to  today’s  session  I 
would  describe  my 
relationship  with  at  least  one 
of  the  people  in  my  pod  as  a 
friendship. 

o 

o 

o 

o 

o 

Q22 

Prior  to  today’s  session,  I 
have  worked  with  at  least  one 
of  the  people  in  my  pod. 

o 

o 

o 

o 

o 

Q24 

There  was  enough  time 
allotted  to  solve  the  problems 
during  the  task. 

o 

o 

o 

o 

o 

Q25 

The  information  and 
instructions  given  by  the 
session  facilitator  were  clear. 

o 

o 

o 

o 

o 

Q26 

The  task  was  difficult  to 
complete. 

o 

o 

o 

o 

o 
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Within  POD  Participation  and  Perspectives  -  Task  Questionnaire 


Please  reflect  on  the  participation  of  the  various  people  in  your  pod  during  the  task  that  you 
just  completed.  Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a 
scale  of  0  to  4  (0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Q27 

The  outcome  of  the  task 
reflects  the  input  from  all  the 
people  in  my  pod. 

o 

o 

o 

O 

o 

Q28 

The  differences  of  opinion 
within  my  pod  helped  us 
reach  better  decisions  during 
the  task. 

o 

o 

o 

o 

o 

Q29 

The  people  in  my  pod 
actively  participated  in  the 
problem  solving  process. 

o 

o 

o 

o 

o 

Q30 

The  people  in  my  pod 
communicated  effectively 
with  one  another. 

o 

o 

o 

o 

o 

Q31 

The  people  in  my  pod  were 
engaged  in  the  decision 
making  process. 

o 

o 

o 

o 

o 

Q32 

The  people  in  my  pod 
generated  various  alternative 
ideas. 

o 

o 

o 

o 

o 

Q33 

A  clear  leader  emerged 
within  my  pod. 

o 

o 

o 

o 

o 

Q34 

There  was  more  than  one 
leader  within  my  pod. 

o 

o 

o 

o 

o 

Q35 

Personality  styles  within  my 
pod  got  in  the  way  of 
completing  the  task. 

o 

o 

o 

o 

o 

Q36 

The  people  within  my  pod 
had  frequent  differences  of 
opinion. 

o 

o 

o 

o 

o 

78  DRDC  CSS  CR  201 1-32 


Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q37 

The  people  in  my  pod  were 
motivated  to  complete  the 
task. 

o 

o 

o 

O 

o 

Q38 

The  people  in  my  pod  were 
able  to  stay  focused  on  the 
task. 

o 

o 

o 

O 

o 
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G.4  Closed  Pod  Task  #2  Questionnaire 


Do  not  complete  this  questionnaire  until  instructed  to  do  so 


Individual  Participation  and  Perspectives  -  Task  Questionnaire 

Please  reflect  on  your  own  participation  and  thoughts  during  the  task  that  you  just  completed. 
Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a  scale  of  0  to  4 
(0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Ql 

I  am  satisfied  with  the 
problem  solving  processes  I 
used  during  the  task. 

o 

o 

o 

O 

o 

Q2 

I  am  satisfied  with  the 
opportunities  I  had  to  provide 
input. 

o 

o 

o 

o 

o 

Q3 

I  am  satisfied  with  the  overall 
quality  of  the  outcome  from 
the  task. 

o 

o 

o 

o 

o 

Q4 

I  am  frustrated  working  with 
the  people  in  my  pod  group. 

o 

o 

o 

o 

o 

Q6 

1  participated  actively  in  the 
decision  making  process. 

o 

o 

o 

o 

o 

Q7 

1  participated  in  a  leadership 
role  within  my  own  pod. 

o 

o 

o 

o 

o 

Q9 

I  facilitated  discussion  within 
my  own  pod. 

o 

o 

o 

o 

o 

Qll 

I  recorded  information  on 
behalf  of  my  own  pod. 

o 

o 

o 

o 

o 

Q13 

I  was  frustrated  by 
differences  of  opinion  within 
my  pod  during  the  task. 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q15 

I  trust  the  people  in  my  pod. 

o 

o 

o 

O 

o 

Q17 

I  felt  a  sense  of  belonging 
within  my  pod. 

o 

o 

o 

o 

o 

Q19 

I  agree  with  the  decisions  and 
outcomes  from  the  task. 

o 

o 

o 

o 

o 

Q24 

There  was  enough  time 
allotted  to  solve  the  problems 
during  the  task. 

o 

o 

o 

o 

o 

Q25 

The  information  and 
instructions  given  by  the 
session  facilitator  were  clear. 

o 

o 

o 

o 

o 

Q26 

The  task  was  difficult  to 
complete. 

o 

o 

o 

o 

o 
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Within  POD  Participation  and  Perspectives  -  Task  Questionnaire 


Please  reflect  on  the  participation  of  the  various  people  in  your  pod  during  the  task  that  you 
just  completed.  Read  each  of  the  following  statements  and  then  rate  your  level  of  agreement  on  a 
scale  of  0  to  4  (0  -  strongly  disagree;  1  -  disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). 


Strongly  Strongly 


disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

agree 

4 

Q27 

The  outcome  of  the  task 
reflects  the  input  from  all  the 
people  in  my  pod. 

o 

o 

o 

O 

o 

Q28 

The  differences  of  opinion 
within  my  pod  helped  us 
reach  better  decisions  during 
the  task. 

o 

o 

o 

o 

o 

Q29 

The  people  in  my  pod 
actively  participated  in  the 
problem  solving  process. 

o 

o 

o 

o 

o 

Q30 

The  people  in  my  pod 
communicated  effectively 
with  one  another. 

o 

o 

o 

o 

o 

Q31 

The  people  in  my  pod  were 
engaged  in  the  decision 
making  process. 

o 

o 

o 

o 

o 

Q32 

The  people  in  my  pod 
generated  various  alternative 
ideas. 

o 

o 

o 

o 

o 

Q33 

A  clear  leader  emerged 
within  my  pod. 

o 

o 

o 

o 

o 

Q34 

There  was  more  than  one 
leader  within  my  pod. 

o 

o 

o 

o 

o 

Q35 

Personality  styles  within  my 
pod  got  in  the  way  of 
completing  the  task. 

o 

o 

o 

o 

o 

Q36 

The  people  within  my  pod 
had  frequent  differences  of 
opinion. 

o 

o 

o 

o 

o 
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Strongly 

disagree 

0 

Disagree 

1 

Neutral 

2 

Agree 

3 

Strongly 

agree 

4 

Q37 

The  people  in  my  pod  were 
motivated  to  complete  the 
task. 

o 

o 

o 

o 

o 

Q38 

The  people  in  my  pod  were 
able  to  stay  focused  on  the 
task. 

o 

o 

o 

o 

o 
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Annex  H  Background  Information  Questionnaire 


1.  What  is  your  current  job  title?  _ 

2.  How  long  have  your  worked  in  this  position? _ months _ years 

3.  How  long  have  you  worked  for  this  organization?  _ months _ years 

4.  How  long  have  you  worked  in  emergency  management? _ months _ 

years 

5.  What  are  your  main  roles  and  responsibilities  with  respect  to: 

a.  Planning  for  emergencies 


b.  Responding  to  emergencies 

6.  What  is  your  academic  background? 

a.  College  diplomas: 

b.  University  degrees: 

7.  What  is  your  current  age?  _ years 

8.  Gender: 

9.  Which  language(s)  are  you  fluent  enough  in  to  work  in?  Check  all  that  apply. 

□  French 
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□  English 

□  Other 

If  Other,  please  list: 
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Annex  I  Confidentiality  /  Intellectual  Property 
Agreement 


Subject:  Agreement  for  the  University  of  Ottawa  project  entitled  “In  Vivo  Simulation 

of  Meta-Organizational  Shared  Decision  Making”  (the  “Project”) 

In  order  to  protect  confidentiality  of  materials  created  and  data  collected  over  the  course  of  the 

“In  Vivo  Simulation  of  Meta-Organizational  Shared  Decision  Making”  project,  we  are  kindly 

asking  reviewers  of  materials  to  agree  to  the  confidentiality  and  intellectual  property  clauses 

described  below. 

Confidentiality: 

1 . 1  “Confidential  Information”  means  all  data  and  information  relating  to  the  research 
project. 

1.2  I  understand  and  agree  that,  as  part  of  my  participation,  I  may  have  access  to 
Confidential  Information  belonging  to  the  University  of  Ottawa  or  other  participants 
including,  without  limitation,  the  results  of  this  project.  I  will  regard  such  Confidential 
Information  as  strictly  confidential  and  agree  not  to  reveal  any  such  Confidential 
Information  to  any  other  person,  firm,  coiporation,  company  or  entity  unless  specifically 
authorized  to  do  so  in  writing  by  Louise  Lemyre,  the  Principal  Investigator  of  the 
Project.  This  obligation  of  secrecy  will  continue  after  my  participation  in  the  Project  has 
ended. 

Intellectual  Property: 

2.1  “Intellectual  Property”  includes,  but  is  not  limited  to:  technical  information,  know-how, 
copyrights,  patents,  models,  patterns,  drawings,  specification,  prototypes,  and  inventions. 

2.2  I  agree  to  assign  ownership  of  all  Intellectual  Property  I  contribute  to  create,  develop  or 
produce  as  part  of  my  participation  in  the  Project  to  the  University  of  Ottawa.  I  agree  to 
execute  such  additional  conveyances  and  other  documents  conveying  title  or  copyright  to 
such  Intellectual  Property  to  the  University  of  Ottawa  as  the  University  of  Ottawa  may 
require.  Should  any  Intellectual  Property  assigned  to  the  University  of  Ottawa  be  in  the 
form  of  copyrights,  as  an  author  of  copyrighted  Intellectual  Property,  I  hereby 
permanently  waive  my  moral  rights,  as  defined  in  the  Copyright  Act,  in  respect  such 
Intellectual  Property. 


Signed: 


(Witness) 


Name: 


Date: 
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Annex  J  Participant  ID  &  Information  Linking 
Form 


Participant 

Name: 


Participant 

Label: 


Session  #: 


Session 
Type  (check 
one): 


o  1  University  of  Ottawa 
Students 

o2  Professional  Students 


o3  Senior  Officials 


Pod  #:  o:  One 
o2  Two 
o3  Three 
o4  Four 


Participant 

#: 


o:  One 
o2  Two 
o3  Three 


Participant 

Organization: 


Participant  oa  Military 
Type: 

o2  ICS  Oriented 


o3  Non-ICS  Oriented 


Pod  Status: 

03  Connected 

# 

Oj  One 

Participants 

o2  Closed 

in  Pod: 

o2  Two 

o3  Three 
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Annex  K  Task  Worksheets 


K.1  Open  Pod  Collaboration  Task  Worksheet  #1 

INSTRUCTIONS 


•  First  20  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  You  can  use  the  2-way  radios  or  chat  function  to  communicate  with  the  other 
pods. 

o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Next  35  minutes: 

o  The  video  conference  line  will  be  open  with  the  other  pods, 
o  Continue  to  work  on  the  tasks  outlined  in  the  worksheet, 
o  Continue  to  record  your  responses  directly  on  the  worksheet. 

•  Final  15  minutes: 

o  When  directed,  complete  the  questionnaire  that  was  included  in  the  envelope, 
o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 

BACKGROUND  INFORMATION  FOR  TASK 


•  Over  the  past  24  hours,  there  have  been  media  reports  that  martial  law  and  curfews  are 
going  to  be  applied.  Rumours  are  circulating  that  special  legislation  or  the  Emergencies 
Act  will  be  declared,  giving  military  greater  control  of  the  city  and  any  movement  in  or 
out  of  it. 

•  While  there  is  no  truth  to  any  of  these  rumours,  there  are  armed  and  protected  soldiers 
who  are  guarding  various  sites  and  moving  throughout  the  city  to  detect  additional 
sources  of  contamination;  sightings  of  these  soldiers  are  adding  fuel  to  rumours. 

•  As  a  result,  nervous  citizens  are  evacuating  prematurely,  ignoring  instructions  to  shelter 
in  place. 

•  There  are  increasing  levels  of  criticism  and  concern  from  both  citizens  and  local  political 
representatives  that  these  communication  issues  have  not  been  adequately  addressed. 

•  Criticisms  include  conflicting  instructions  to  the  public,  messages  that  are  vague  and 
unclear,  and  major  gaps  in  information. 

•  In  order  to  address  these  criticisms,  please  work  both  within  your  own  pod  and  with  the 
other  two  pods  for  the  following  tasks  outlined  in  the  worksheet. 
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1)  Come  to  a  consensus  across  all  organizations  within  all  the  pods  on  the  two  priority 
messages  or  pieces  of  information  that  the  group  feels  it  is  most  important  to  relay  to 
the  general  public  at  this  point.  Record  these  messages  in  the  form  below. 


MESSAGE  #1: 


MESSAGE  #2: 
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2)  In  consultation  with  the  other  organizations  in  all  the  pods,  identify  two  potential 
options  for  approaches  that  could  be  used  by  the  group  to  effectively  deliver  each  of 
the  two  priority  messages  to  the  public.  In  developing  approaches,  consider  who  will 
deliver  the  message,  when  it  can  be  delivered,  what  venue  or  method  would  be 
effective,  and  considerations  to  take  in  account  if  the  option  was  to  be  selected. 


MESSAGE  #1  -  OPTION  #1 
Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 


MESSAGE  #1  -  OPTION  #2 
Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 
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MESSAGE  #2  -  OPTION  #1 

Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 


MESSAGE  #2  -  OPTION  #2 

Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 
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3)  As  a  group,  make  decisions  on  which  option  for  each  message  is  most  appropriate. 


MESSAGE  #1  DECISION 

Decision: 


Rationale  for  decision: 


MESSAGE  #2  DECISION 

Decision: 


Rationale  for  decision: 
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K.2  Open  Pod  Collaboration  Task  Worksheet  #2 

INSTRUCTIONS 


•  First  20  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  You  can  use  the  2-way  radios  or  chat  function  to  communicate  with  the  other 
pods. 

o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Next  35  minutes: 

o  The  video  conference  line  will  be  open  with  the  other  pods, 
o  Continue  to  work  on  the  tasks  outlined  in  the  worksheet, 
o  Continue  to  record  your  responses  directly  on  the  worksheet. 

•  Final  15  minutes: 

o  When  directed,  complete  the  questionnaire  that  was  included  in  the  envelope, 
o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 


BACKGROUND  INFORMATION  FOR  TASK 


•  Responders  travelling  from  other  jurisdictions,  contractors,  volunteers  as  well  as  the 
various  unions  for  the  professional  responders  are  all  questioning  whether  or  not  they 
are  being  adequately  protected  and  what  standards  for  personal  protective  equipment 
should  be  imposed. 

•  Differing  protocols  and  equipment  are  being  used  in  the  same  sites,  while  multiple  and 
shifting  hot  zones  are  complicating  matters  further. 

•  Another  hotly  debated  issue  is  the  access  to  Prussian  Blue  for  responders  and  their 
families. 

•  Some  members  of  the  public  feel  that  there  is  preferential  treatment  for  these  groups, 
while  those  with  access  are  raising  questions  about  the  side  effects  and  long  term 
consequences. 

•  Occupational  health  and  safety  for  traditional  and  non-traditional  responders  has 
become  a  major  issue  in  the  context  of  the  scenario,  both  in  the  short  term  and  in  the 
longer  term. 


DRDC  CSS  CR  201 1-32 


93 


In  order  to  address  these  concerns,  please  work  both  within  your  own  pod  and  with  the 
other  two  pods  for  the  following  tasks  outlined  in  the  worksheet. 
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1)  Come  to  a  consensus  across  all  organizations  within  all  the  pods  on  the  two  most 

significant  responder  health  and  safety  issues  that  you  feel  are  most  important  to  address. 
Record  these  issues  in  the  form  below. 


HEALTH  &  SAFETY  ISSUE  #1: 


HEALTH  &  SAFETY  ISSUE  #2: 
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2)  In  consultation  with  the  other  organizations  in  all  pods,  identify  two  potential  options  for 
approaches  that  could  be  used  by  the  group  to  effectively  address  the  two  health  &  safety 
issues.  In  developing  approaches,  consider  who  will  deliver  the  interventions,  when  it  can 
be  delivered,  what  method  would  be  effective,  and  considerations  to  take  in  account  if 
the  option  was  to  be  selected.  Describe  these  approaches  in  the  forms  below. 


HEALTH  &  SAFETY  ISSUE  #1  -  OPTION  #1 
Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 


HEALTH  &  SAFETY  ISSUE  #1  -  OPTION  #2 
Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 
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HEALTH  &  SAFETY  ISSUE  #2  -  OPTION  #1 

Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 


HEALTH  &  SAFTEY  ISSUE  #2  -  OPTION  #2 

Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 
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3)  As  a  group,  make  decisions  on  which  option  for  each  health  and  safety  issue  is  most 
appropriate.  Record  the  decisions  and  rationale  in  the  grid  below. 


HEALTH  &  SAFETY  ISSUE  #1: 
Decision  on  which  option  to  act  on: 


Rationale  for  the  decision: 


HEALTH  &  SAFETY  ISSUE  #2: 
Decision  on  which  option  to  act  on: 


Rationale  for  the  decision: 
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K.3  Open  Pod  Coordination  Task  Worksheet  #1 

INSTRUCTIONS 


•  First  20  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  You  can  use  the  2-way  radios  or  chat  function  to  communicate  with  the  other 
pods. 

o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Next  35  minutes: 

o  The  video  conference  line  will  be  open  with  the  other  pods, 
o  Continue  to  work  on  the  tasks  outlined  in  the  worksheet, 
o  Continue  to  record  your  responses  directly  on  the  worksheet. 

•  Final  15  minutes: 

o  When  directed,  complete  the  questionnaire  that  was  included  in  the  envelope, 
o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 


BACKGROUND  INFORMATION  FOR  TASK 


•  Over  the  past  24  hours,  there  have  been  media  reports  that  martial  law  and  curfews  are 
going  to  be  applied.  Rumours  are  circulating  that  special  legislation  or  the  Emergencies 
Act  will  be  declared,  giving  military  greater  control  of  the  city  and  any  movement  in  or 
out  of  it. 

•  While  there  is  no  truth  to  any  of  these  rumours,  there  are  armed  and  protected  soldiers 
who  are  guarding  various  sites  and  moving  throughout  the  city  to  detect  additional 
sources  of  contamination;  sightings  of  these  soldiers  are  adding  fuel  to  rumours. 

•  As  a  result,  nervous  citizens  are  evacuating  prematurely,  ignoring  instructions  to  shelter 
in  place. 

•  There  are  increasing  levels  of  criticism  and  concern  from  both  citizens  and  local  political 
representatives  that  these  communication  issues  have  not  been  adequately  addressed. 

•  Criticisms  include  conflicting  instructions  to  the  public,  messages  that  are  vague  and 
unclear,  and  major  gaps  in  information. 

•  In  order  to  address  these  criticisms,  please  work  both  within  your  own  pod  and  with  the 
other  two  pods  for  the  following  tasks  outlined  in  the  worksheet. 
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1)  Determine  the  two  main  messages  or  pieces  of  information  that  you  feel  that  it  is  most 
important  for  you  to  relay  to  the  general  public  at  this  point.  Record  these  messages  in 
the  form  below.  Then  consult  with  others  in  your  pod  and  in  other  pods  to  understand 
their  priority  messages.  Use  the  form  below  to  keep  track  of  their  messages. 


YOUR  PRIORITY  MESSAGES: 
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PRIORITY  MESSAGES  FROM  OTHERS  IN  YOUR  POD 


Organization: 

Message  #1 

Message  #2 

Organization: 

Message  #1 

Message  #2 
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PRIORITY  MESSAGES  FROM  OTHERS  IN  OTHER  PODS 


Organization: 

Message  #1 

Message  #2 

Organization: 

Message  #1 

Message  #2 

Organization: 

Message  #1 

Message  #2 

Organization: 

Message  #1 

Message  #2 

Organization: 

Message  #1 

Message  #2 

Organization: 

Message  #1 

Message  #2 

102 


DRDC  CSS  CR  2011-32 


2)  In  consultation  with  the  other  members  in  all  the  pods,  determine  where  there  are 
overlaps  in  messages,  potential  gaps  or  contradictions  with  respect  to  YOUR  messages. 

Record  the  overlaps  and  gaps  in  the  form  below. 


YOUR  MESSAGE  #1: 


Identified  overlaps,  gaps,  and  contradictions  with  other  members'  messages: 


YOUR  MESSAGE  #2: 


Identified  overlaps,  gaps,  and  contradictions  with  other  members'  messages: 
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3)  Work  with  the  others  in  your  pod  to  determine  options  for  combining  or  grouping 
messages  with  other  members,  and  then  decide  on  the  most  appropriate  grouping  and 
methods  of  delivery  for  YOUR  messages  that  need  to  be  delivered  to  the  public  taking  into 
account  options  identified.  Record  the  options  and  decisions  in  the  grid  below. 


YOUR  MESSAGE  #1: 

Potential  options  for  grouping/combining  with  other  messages : 

Decision  on  grouping  or  not  grouping  with  other  messages 

Rationale  for  decision  on  grouping  or  not  grouping  and  anv  changes  required  to 

message  if  grouping: 

Decision  on  how  message  will  be  delivered  (who,  when,  venue/method) 

YOUR  MESSAGE  #2: 

Potential  options  for  grouping/combining  with  other  messages: 

Decision  on  grouping  or  not  grouping  with  other  messages 

Rationale  for  decision  on  grouping  or  not  grouping  and  anv  changes  required  to 

message  if  grouping: 

Decision  on  how  message  will  be  delivered  (who,  when,  venue/method) 
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K.4  Open  Pod  Coordination  Task  Worksheet  #2 

INSTRUCTIONS 


•  First  20  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  You  can  use  the  2-way  radios  or  chat  function  to  communicate  with  the  other 
pods. 

o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Next  35  minutes: 

o  The  video  conference  line  will  be  open  with  the  other  pods, 
o  Continue  to  work  on  the  tasks  outlined  in  the  worksheet, 
o  Continue  to  record  your  responses  directly  on  the  worksheet. 

•  Final  15  minutes: 

o  When  directed,  complete  the  questionnaire  that  was  included  in  the  envelope, 
o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 


BACKGROUND  INFORMATION  FOR  TASK 


•  Responders  travelling  from  other  jurisdictions,  contractors,  volunteers  as  well  as  the 
various  unions  for  the  professional  responders  are  all  questioning  whether  or  not  they 
are  being  adequately  protected  and  what  standards  for  personal  protective  equipment 
should  be  imposed. 

•  Differing  protocols  and  equipment  are  being  used  in  the  same  sites,  while  multiple  and 
shifting  hot  zones  are  complicating  matters  further. 

•  Another  hotly  debated  issue  is  the  access  to  Prussian  Blue  for  responders  and  their 
families. 

•  Some  members  of  the  public  feel  that  there  is  preferential  treatment  for  these  groups, 
while  those  with  access  are  raising  questions  about  the  side  effects  and  long  term 
consequences. 

•  Occupational  health  and  safety  for  traditional  and  non-traditional  responders  has 
become  a  major  issue  in  the  context  of  the  scenario,  both  in  the  short  term  and  in  the 
longer  term. 
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In  order  to  address  these  concerns,  please  work  both  within  your  own  pod  and  with  the 
other  two  pods  for  the  following  tasks  outlined  in  the  worksheet. 
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1)  Determine  the  two  most  significant  responder  health  and  safety  issues  that  you  feel  are 
most  important  for  you  to  address.  Record  these  issues  in  the  form  below.  Then  consult 
with  others  in  your  pod  and  in  other  pods  to  understand  their  priority  issues.  Use  the 
form  below  to  keep  track  of  their  issues. 

YOUR  PRIORITY  HEALTH  &  SAFETY  ISSUES: 


HEALTH  &  SAFETY  ISSUE  #1: 


HEALTH  &  SAFETY  ISSUE  #2: 
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PRIORITY  HEALTH  &  SAFETY  ISSUES  FROM  OTHERS  IN  YOUR 

POD 


Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

108 


DRDC  CSS  CR  2011-32 


PRIORITY  HEALTH  &  SAFETY  ISSUES  FROM  OTHERS  IN 

OTHER  PODS 


Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

DRDC  CSS  CR  201 1-32 


109 


2)  In  consultation  with  the  other  members  in  all  the  pods,  determine  where  there  are 

overlaps,  potential  gaps  or  contradictions  with  respect  to  YOUR  health  and  safety  issues. 

Record  the  overlaps  and  gaps  in  the  form  below. 


YOUR  HEALTH  &  SAFETY  ISSUE  #1: 

Identified  overlaps,  gaps,  and  contradictions  with  other  members'  health  and  safety 
issues: 


YOUR  HEALTH  &  SAFETY  ISSUE  #2: 

Identified  overlaps,  gaps,  and  contradictions  with  other  members'  health  and  safety 
issues: 
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3)  Work  with  the  other  members  to  decide  on  a  plan  for  following  up  on  these  health  and 
safety  concerns.  Describe  the  actions  that  need  to  be  taken  for  YOUR  health  and  safety 
issues  to  be  addressed,  taking  into  account  options  identified.  Record  the  options  and 
decisions  in  the  grid  below. 


YOUR  HEALTH  &  SAFETY  ISSUE  #1: 

Potential  options  for  grouping  or  combining  actions  to  be  taken  with  other  members: 


Decision  on  grouping  or  not  grouping  actions  with  other  members: 


Rationale  for  the  decision  on  grouping/not  grouping  and  any  changes  to  actions 
required  if  grouping: 


Decision  on  which  actions  will  be  implemented:  (who,  when,  what  methods) 


YOUR  HEALTH  &  SAFETY  ISSUE  #2: 

Potential  options  for  grouping  or  combining  actions  to  be  taken  with  other  members: 


Decision  on  grouping  or  not  grouping  actions  with  other  members: 


Rationale  for  the  decision  on  grouping/not  grouping  and  any  changes  to  actions 
required  if  grouping: 


Decision  on  which  actions  will  be  implemented:  (who,  when,  what  methods) 


DRDC  CSS  CR  201 1-32 


111 


K.5  Closed  Pod  Collaboration  Task  Worksheet  #1 

INSTRUCTIONS 


•  First  55  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Final  15  minutes: 

o  When  instructed,  complete  the  questionnaire  that  was  included  in  the 
envelope. 

o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 


BACKGROUND  INFORMATION  FOR  TASK 


•  Over  the  past  24  hours,  there  have  been  media  reports  that  martial  law  and  curfews  are 
going  to  be  applied.  Rumours  are  circulating  that  special  legislation  or  the  Emergencies 
Act  will  be  declared,  giving  military  greater  control  of  the  city  and  any  movement  in  or 
out  of  it. 

•  While  there  is  no  truth  to  any  of  these  rumours,  there  are  armed  and  protected  soldiers 
who  are  guarding  various  sites  and  moving  throughout  the  city  to  detect  additional 
sources  of  contamination;  sightings  of  these  soldiers  are  adding  fuel  to  rumours. 

•  As  a  result,  nervous  citizens  are  evacuating  prematurely,  ignoring  instructions  to  shelter 
in  place. 

•  There  are  increasing  levels  of  criticism  and  concern  from  both  citizens  and  local  political 
representatives  that  these  communication  issues  have  not  been  adequately  addressed. 

•  Criticisms  include  conflicting  instructions  to  the  public,  messages  that  are  vague  and 
unclear,  and  major  gaps  in  information. 

•  In  order  to  address  these  criticisms,  please  work  within  your  pod  for  the  following  tasks 
outlined  in  the  worksheet. 
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1)  Come  to  a  consensus  across  aM  members  within  the  pod  on  the  two  priority  messages  or 
pieces  of  information  that  the  group  feels  it  is  most  important  to  relay  to  the  general 
public  at  this  point.  Record  these  messages  in  the  form  below. 


MESSAGE  #1: 


MESSAGE  #2: 
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2)  In  consultation  with  the  other  members  in  the  pod,  identify  two  potential  options  for 
approaches  that  could  be  used  by  the  group  to  effectively  deliver  each  of  the  two  priority 
messages  to  the  public.  In  developing  approaches,  consider  who  will  deliver  the  message, 
when  it  can  be  delivered,  what  venue  or  method  would  be  effective,  and  considerations  to 
take  in  account  if  the  option  was  to  be  selected. 


MESSAGE  #1  -  OPTION  #1 
Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 


MESSAGE  #1  -  OPTION  #2 
Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 
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MESSAGE  #2  -  OPTION  #1 

Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 


MESSAGE  #2  -  OPTION  #2 

Main  messenger: 


Timing  of  message: 


Venue/method: 


Considerations: 
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3)  As  a  group,  make  decisions  on  which  option  for  each  message  is  most  appropriate. 


MESSAGE  #1  DECISION 

Decision: 


Rationale  for  decision: 


MESSAGE  #2  DECISION 

Decision: 


Rationale  for  decision: 
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K.6  Closed  Pod  Collaboration  Task  Worksheet  #2 

INSTRUCTIONS 


•  First  55  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Final  15  minutes: 

o  When  instructed,  complete  the  questionnaire  that  was  included  in  the 
envelope. 

o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 


BACKGROUND  INFORMATION  FOR  TASK 


•  Responders  travelling  from  other  jurisdictions,  contractors,  volunteers  as  well  as  the 
various  unions  for  the  professional  responders  are  all  questioning  whether  or  not  they 
are  being  adequately  protected  and  what  standards  for  personal  protective  equipment 
should  be  imposed. 

•  Differing  protocols  and  equipment  are  being  used  in  the  same  sites,  while  multiple  and 
shifting  hot  zones  are  complicating  matters  further. 

•  Another  hotly  debated  issue  is  the  access  to  Prussian  Blue  for  responders  and  their 
families. 

•  Some  members  of  the  public  feel  that  there  is  preferential  treatment  for  these  groups, 
while  those  with  access  are  raising  questions  about  the  side  effects  and  long  term 
consequences. 

•  Occupational  health  and  safety  for  traditional  and  non-traditional  responders  has 
become  a  major  issue  in  the  context  of  the  scenario,  both  in  the  short  term  and  in  the 
longer  term. 

•  In  order  to  address  these  concerns,  please  work  within  your  pod  for  the  following  tasks 
outlined  in  the  worksheet. 
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4)  Come  to  a  consensus  across  aM  members  within  the  pod  on  the  two  most  significant 
responder  health  and  safety  issues  that  you  feel  are  most  important  to  address.  Record 
these  issues  in  the  form  below. 


HEALTH  &  SAFETY  ISSUE  #1: 


HEALTH  &  SAFETY  ISSUE  #2: 


118 


DRDC  CSS  CR  2011-32 


5)  In  consultation  with  the  other  members  in  the  pod,  identify  two  potential  options  for 
approaches  that  could  be  used  by  the  group  to  effectively  address  the  two  health  &  safety 
issues.  In  developing  approaches,  consider  who  will  deliver  the  interventions,  when  it  can 
be  delivered,  what  method  would  be  effective,  and  considerations  to  take  in  account  if 
the  option  was  to  be  selected.  Describe  these  approaches  in  the  forms  below. 


HEALTH  &  SAFETY  ISSUE  #1  -  OPTION  #1 
Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 


HEALTH  &  SAFETY  ISSUE  #1  -  OPTION  #2 
Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 
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HEALTH  &  SAFETY  ISSUE  #2  -  OPTION  #1 

Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 


HEALTH  &  SAFTEY  ISSUE  #2  -  OPTION  #2 

Who  will  deliver  the  interventions: 


Timing: 


Method: 


Considerations: 
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6)  As  a  group,  make  decisions  on  which  option  for  each  health  and  safety  issue  is  most 
appropriate.  Record  the  decisions  and  rationale  in  the  grid  below. 


HEALTH  &  SAFETY  ISSUE  #1: 
Decision  on  which  option  to  act  on: 


Rationale  for  the  decision: 


HEALTH  &  SAFETY  ISSUE  #2: 
Decision  on  which  option  to  act  on: 


Rationale  for  the  decision: 
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K.7  Closed  Pod  Coordination  Task  Worksheet  #1 

INSTRUCTIONS 


•  First  55  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Final  15  minutes: 

o  When  instructed,  complete  the  questionnaire  that  was  included  in  the 
envelope. 

o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 


BACKGROUND  INFORMATION  FOR  TASK 


•  Over  the  past  24  hours,  there  have  been  media  reports  that  martial  law  and  curfews  are 
going  to  be  applied.  Rumours  are  circulating  that  special  legislation  or  the  Emergencies 
Act  will  be  declared,  giving  military  greater  control  of  the  city  and  any  movement  in  or 
out  of  it. 

•  While  there  is  no  truth  to  any  of  these  rumours,  there  are  armed  and  protected  soldiers 
who  are  guarding  various  sites  and  moving  throughout  the  city  to  detect  additional 
sources  of  contamination;  sightings  of  these  soldiers  are  adding  fuel  to  rumours. 

•  As  a  result,  nervous  citizens  are  evacuating  prematurely,  ignoring  instructions  to  shelter 
in  place. 

•  There  are  increasing  levels  of  criticism  and  concern  from  both  citizens  and  local  political 
representatives  that  these  communication  issues  have  not  been  adequately  addressed. 

•  Criticisms  include  conflicting  instructions  to  the  public,  messages  that  are  vague  and 
unclear,  and  major  gaps  in  information. 

•  In  order  to  address  these  criticisms,  please  work  within  your  pod  for  the  following  tasks 
outlined  in  the  worksheet. 
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4)  Determine  the  two  main  messages  or  pieces  of  information  that  you  feel  that  it  is  most 
important  for  you  to  relay  to  the  general  public  at  this  point.  Record  these  messages  in 
the  form  below.  Then  consult  with  others  in  your  pod  to  understand  their  priority 
messages.  Use  the  form  below  to  keep  track  of  their  messages. 

YOUR  PRIORITY  MESSAGES: 


MESSAGE  #1: 


MESSAGE  #2: 


PRIORITY  MESSAGES  FROM  OTHERS  IN  YOUR  POD 


Organization: 

Message  #1 

Message  #2 

Organization: 

Message  #1 

Message  #2 
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5)  In  consultation  with  the  other  members  in  your  pod,  determine  where  there  are  overlaps 
in  messages,  potential  gaps  or  contradictions  with  respect  to  YOUR  messages.  Record  the 
overlaps  and  gaps  in  the  form  below. 


YOUR  MESSAGE  #1: _ 

Identified  overlaps,  gaps,  and  contradictions  with  other  members'  messages: 


YOUR  MESSAGE  #2: _ 

Identified  overlaps,  gaps,  and  contradictions  with  other  members'  messages: 
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6)  Work  with  the  other  members  in  the  pod  to  determine  options  for  combining  or  grouping 
the  messages  from  the  other  pod  members,  and  then  decide  on  the  most  appropriate 
grouping  and  methods  of  delivery  for  YOUR  messages  that  need  to  be  delivered  to  the 
public  taking  into  account  options  identified.  Record  the  options  and  decisions  in  the  grid 
below. 


YOUR  MESSAGE  #1: 

Potential  options  for  grouping/combining  with  other  messages: 

Decision  on  grouping  or  not  grouping  with  other  messages 

Rationale  for  decision  on  grouping  or  not  grouping  and  anv  changes  required  to  message  if 

grouping: 

Decision  on  how  message  will  be  delivered  (who,  when,  venue/method) 

YOUR  MESSAGE  #2: 

Potential  options  for  grouping/combining  with  other  messages: 

Decision  on  grouping  or  not  grouping  with  other  messages 

Rationale  for  decision  on  grouping  or  not  grouping  and  anv  changes  required  to  message 

if  grouping: 

Decision  on  how  message  will  be  delivered  (who,  when,  venue/method) 
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K.8  Closed  Pod  Coordination  Task  Worksheet  #2 

INSTRUCTIONS 


•  First  55  minutes: 

o  Work  with  the  members  of  your  pod  on  the  tasks  outlined  in  this  worksheet, 
o  Please  record  your  responses  directly  on  the  worksheet  -  these  will  be  collected 
at  the  end  of  the  session. 

•  Final  15  minutes: 

o  When  instructed,  complete  the  questionnaire  that  was  included  in  the 
envelope. 

o  Your  completed  questionnaire  will  be  collected  at  the  end  of  the  session. 


BACKGROUND  INFORMATION  FOR  TASK 


•  Responders  travelling  from  other  jurisdictions,  contractors,  volunteers  as  well  as  the 
various  unions  for  the  professional  responders  are  all  questioning  whether  or  not  they 
are  being  adequately  protected  and  what  standards  for  personal  protective  equipment 
should  be  imposed. 

•  Differing  protocols  and  equipment  are  being  used  in  the  same  sites,  while  multiple  and 
shifting  hot  zones  are  complicating  matters  further. 

•  Another  hotly  debated  issue  is  the  access  to  Prussian  Blue  for  responders  and  their 
families. 

•  Some  members  of  the  public  feel  that  there  is  preferential  treatment  for  these  groups, 
while  those  with  access  are  raising  questions  about  the  side  effects  and  long  term 
consequences. 

•  Occupational  health  and  safety  for  traditional  and  non-traditional  responders  has 
become  a  major  issue  in  the  context  of  the  scenario,  both  in  the  short  term  and  in  the 
longer  term. 

•  In  order  to  address  these  concerns,  please  work  within  your  pod  for  the  following  tasks 
outlined  in  the  worksheet. 
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4)  Determine  the  two  most  significant  responder  health  and  safety  issues  that  you  feel  are 
most  important  for  you  to  address.  Record  these  issues  in  the  form  below.  Then  consult 
with  others  in  your  pod  to  understand  their  priority  issues.  Use  the  form  below  to  keep 
track  of  their  issues. 

YOUR  PRIORITY  HEALTH  &  SAFETY  ISSUES: 


HEALTH  &  SAFETY  ISSUE  #1: 


HEALTH  &  SAFETY  ISSUE  #2: 


PRIORITY  HEALTH  &  SAFETY  ISSUES  FROM  OTHERS  IN  YOUR 

POD 


Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 

Organization: 

Health  &  Safety  Issue  #1 

Health  &  Safety  Issue  #2 
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5)  In  consultation  with  the  other  members  in  your  pod,  determine  where  there  are  overlaps, 
potential  gaps  or  contradictions  with  respect  to  YOUR  health  and  safety  issues.  Record 

the  overlaps  and  gaps  in  the  form  below. 


YOUR  HEALTH  &  SAFETY  ISSUE  #1: _ 

Identified  overlaps,  gaps,  and  contradictions  with  other  members'  health  and  safety  issues: 


YOUR  HEALTH  &  SAFETY  ISSUE  #2: _ 

Identified  overlaps,  gaps,  and  contradictions  with  other  members'  health  and  safety  issues: 
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6)  Work  with  the  other  members  to  decide  on  a  plan  for  following  up  on  these  health  and 
safety  concerns.  Describe  the  actions  that  need  to  be  taken  for  YOUR  health  and  safety 
issues  to  be  addressed,  taking  into  account  options  identified.  Record  the  options  and 
decisions  in  the  grid  below. 


YOUR  HEALTH  &  SAFETY  ISSUE  #1: _ 

Potential  options  for  grouping  or  combining  actions  to  be  taken  with  other  members: 


Decision  on  grouping  or  not  grouping  actions  with  other  members: 


Rationale  for  the  decision  on  grouping/not  grouping  and  any  changes  to  actions  required  if 
grouping: 


Decision  on  which  actions  will  be  implemented:  (who,  when,  what  methods) 


YOUR  HEALTH  &  SAFETY  ISSUE  #2: _ 

Potential  options  for  grouping  or  combining  actions  to  be  taken  with  other  members: 


Decision  on  grouping  or  not  grouping  actions  with  other  members: 


Rationale  for  the  decision  on  grouping/not  grouping  and  any  changes  to  actions  required  if 
grouping: 


Decision  on  which  actions  will  be  implemented:  (who,  when,  what  methods) 
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Annex  L  Consistency  of  Interpretation  Instrument 


Universite  d’Ottawa 

Institul  de  recherche  sur  la 
wnte  des  populations 


University  of  Ottawa 

Institute  of 
Population  Health 


This  questionnaire  is  designed  to  assess  the  scenarios  with  respect  to  which  facts  can  be  recalled. 
Please  select  the  best  answer  for  each  question.  Circle  one  response  for  each  question. 

1)  Where  did  the  initial  package  explode? 

a.  Hospital  loading  dock 

b.  International  Borders  and  Customs  warehouse 

c.  Sort  and  Send  Mail  Delivery  warehouse 

d.  GAP  TV  Station 

2)  Approximately  how  many  people  arrived  at  the  examination  centers  set  up  in  the 
hospital  parking  lot  within  the  first  24  hours? 

a.  10,000 

b.  5,000 

c.  1,000 

d.  200 

3)  Within  24  hours,  the  radius  evacuated  around  the  explosion  was: 

a.  500m 

b.  2km 

c.  10km 

d.  20km 

4)  The  bomb  was  assembled: 

a.  In  a  suburban  house 

b.  In  a  small  town  outside  of  GAPville 

c.  In  the  parking  lot  by  the  hospital 

d.  In  a  van  outside  the  warehouse 
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5)  The  number  of  houses  that  need  to  be  decontaminated  or  destroyed: 

a.  20 

b.  100 

c.  300 

d.  500 

6)  The  spokesperson  for  the  situation  report,  Mr.  Robert  Brights,  has  the  title: 

a.  National  Emergency  Manager,  CBRN  Team 

b.  Chief  of  Police,  Gapville 

c.  Chief  of  Liaison,  City  of  Gapville 

d.  Director  of  Emergency  Management,  Unified  Coordinating  Command 

7)  Additional  chemical  protective  gear  for  responders  and  volunteers  was  obtained  from: 

a.  Red  Cross 

b.  Pulp  and  Paper  Mill 

c.  National  CBRN  Team 

d.  National  Firefighters  Association 

8)  What  was  the  intended  target  for  the  bomb? 

a.  Hospital 

b.  Entertainment  district 

c.  Airport 

d.  GAP  TV  Station 

9)  At  the  48-hour  mark,  which  of  the  following  modes  of  transport  are  still  partially 
operational? 

a.  Light  rail 

b.  Bus  lines 

c.  Airport 

10)  Which  of  the  following  sites  is  not  contaminated: 

a.  South  End  LRT  Station 

b.  Airport 

c.  Mosque 

d.  Johnson  Estates 
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Annex  M  Scenario  Assessment  Guide  Contents 


M.1  Scenario  Assessment  Guide:  Scenario  Complexity  Rating 
Instructions 

Estimated  Time  to  Complete:  2  hours  (40  minutes  per  scenario) 

Steps: _ 

1 .  Please  review  the  Information  and  Consent  Form  located  on  the  following  page.  If  you 
agree  to  its  contents,  please  sign  the  form. 

2.  Fill  out  the  Background  Information  Questionnaire. 

3.  Read  the  definitions  given  for  simple,  complicated  and  complex. 

4.  Read  the  first  scenario  provided  in  this  guide. 

5.  Considering  the  scenario,  fill  out  the  Scenario  Assessment  Questionnaire  for  the  first 
scenario.  Note:  for  ease  of  reference,  you  may  want  to  remove  the  questionnaire  from  the 
package  and  place  beside  the  scenario. 

6.  Read  the  second  scenario  provided,  and  complete  the  questionnaire. 

7.  Repeat  with  the  third  and  final  scenario. 

8.  Return  all  scenarios  and  completed  questionnaires  to  the  facilitator. 

Thank  you  for  your  assistance. 
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M.2  Scenario  Assessment  Guide:  Confidentiality  /  Intellectual 
Property  Agreement 


Subject:  Agreement  for  the  University  of  Ottawa  project  entitled  “In  Vivo  Simulation 

of  Meta-Organizational  Shared  Decision  Making”  (the  “Project”) 

In  order  to  protect  confidentiality  of  materials  created  and  data  collected  over  the  course  of  the 

“In  Vivo  Simulation  of  Meta-Organizational  Shared  Decision  Making”  project,  we  are  kindly 

asking  reviewers  of  materials  to  agree  to  the  confidentiality  and  intellectual  property  clauses 

described  below. 

Confidentiality: 

1 . 1  “Confidential  Information”  means  all  data  and  information  relating  to  the  research 
project. 

1.2  I  understand  and  agree  that,  as  part  of  my  participation,  I  may  have  access  to 
Confidential  Information  belonging  to  the  University  of  Ottawa  or  other  participants 
including,  without  limitation,  the  results  of  this  project.  I  will  regard  such  Confidential 
Information  as  strictly  confidential  and  agree  not  to  reveal  any  such  Confidential 
Information  to  any  other  person,  firm,  corporation,  company  or  entity  unless  specifically 
authorized  to  do  so  in  writing  by  Louise  Lemyre,  the  Principal  Investigator  of  the 
Project.  This  obligation  of  secrecy  will  continue  after  my  participation  in  the  Project  has 
ended. 

Intellectual  Property: 

2.1  “Intellectual  Property”  includes,  but  is  not  limited  to:  technical  information,  know-how, 
copyrights,  patents,  models,  patterns,  drawings,  specification,  prototypes,  and  inventions. 

2.2  I  agree  to  assign  ownership  of  all  Intellectual  Property  I  contribute  to  create,  develop  or 
produce  as  part  of  my  participation  in  the  Project  to  the  University  of  Ottawa.  I  agree  to 
execute  such  additional  conveyances  and  other  documents  conveying  title  or  copyright  to 
such  Intellectual  Property  to  the  University  of  Ottawa  as  the  University  of  Ottawa  may 
require.  Should  any  Intellectual  Property  assigned  to  the  University  of  Ottawa  be  in  the 
form  of  copyrights,  as  an  author  of  copyrighted  Intellectual  Property,  I  hereby 
permanently  waive  my  moral  rights,  as  defined  in  the  Copyright  Act,  in  respect  such 
Intellectual  Property. 


Signed: 


Name: 


Date: 


(Witness) 
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M.3  Scenario  Assessment  Guide:  Background  Information 
Questionnaire 

1 0.  What  is  your  current  job  title?  _ 

1 1 .  How  long  have  your  worked  in  this  position? _ months _ years 

12.  How  long  have  you  worked  for  this  organization?  _ months _ years 

13.  How  long  have  you  worked  in  emergency  management? _ months _ 

years 

14.  What  are  your  main  roles  and  responsibilities  with  respect  to: 

a.  Planning  for  emergencies 


b.  Responding  to  emergencies 


15.  What  is  your  academic  background? 

a.  College  diplomas: 

b.  University  degrees: 


16.  What  is  your  current  age?  _ years 

17.  Gender: 

18.  Which  language(s)  are  you  fluent  enough  in  to  work  in?  Check  all  that  apply. 

□  French 

□  English 

□  Other 

If  Other,  please  list: 
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M.4  Scenario  Assessment  Guide:  Definitions  of  Simple, 
Complicated  and  Complex 


Simple,  complicated  and  complex  can  be  conceptualized  as  a  continuum  ranging  from  simple  to 
complex,  with  complicated  located  at  the  midpoint  of  the  continuum. 

Simple  ->  Complicated  ->  Complex 

Simple  situations  can  be  defined  as  those  that  are  well  understood,  and  involve  a  minimal  number 
of  organizations  to  resolve  satisfactorily.  A  simple  situation  is  well  understood  and  thus  each 
organization  follows  standard  operating  procedures.  In  simple  situations,  solutions  are  known  and 
the  scope  of  the  situation  is  within  an  organization’s  capability. 

Complicated  situations  are  characterized  as  generally  well  understood,  but  they  involve  multiple 
interrelated  organizations.  One  main  difference  between  simple  and  complicated  situations  is  the 
number  of  organizations  required  to  resolve  the  situation,  and  the  dynamic  nature  of  the  situation. 
The  main  similarity  is  that  the  situation  is  relatively  well  understood,  including  the  cause  and 
effect  relationships  between  actions  and  outcomes.  The  situation,  including  the  effect  of 
interventions,  is  relatively  well  understood.  In  complicated  situations,  the  solutions  are  known 
but  the  scope  is  beyond  one  organization’s  capability. 

Complex  situations  involve  considerably  fewer  “knowns”  than  either  simple  or  complicated 
situations.  Due  to  the  high  level  of  uncertainty,  the  cause  and  effect  relationship  between  actions 
and  outcomes  becomes  unclear.  Also  characteristic  of  complex  situations  is  that  standard  actions 
or  procedures  that  might  be  commonly  used  in  less  complex  situations  can  cause  large  ripple 
effects  that  then  further  contribute  to  the  complexity  of  the  situation.  In  complex  situations,  the 
solution  is  partly  unknown  and  the  scope  is  well  beyond  one  organization’s  capability. 
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M.5  Scenario  Assessment  Guide:  Blackout  Scenario 


Wednesday  6:00  am 

Wednesday  morning,  the  province  of  Anticipate  is  covered  in  ice  after  one  night  of 
unexpected  heavy  frozen  rain.  The  hardest  hit  city  of  Gapville  wakes  up  to  over  43  mm 
of  ice  build-up,  strong  winds,  plummeting  temperatures  of  minus  20  degree  Celsius 
weather  and  a  power  outage  across  the  entire  city.  Traffic  lights  aren't  working,  hank 
machines,  point-of-sale  purchases  and  gas  pumps  are  inaccessible.  Public  transit  is 
paralyzed  with  the  weather  conditions  and  the  blackout. 

The  municipal  government  orders  all  snow  removal  teams  onto  the  roads,  but  they  are 
overwhelmed,  and  need  support.  Gapville  Hydro  workers  are  trying  to  find  the  source  of 
the  city-wide  power  outage,  and  are  repairing  the  ice  damaged  lines  as  fast  as  possible. 

Wednesday  12:00  pm 

The  McLaughlin  Memorial  Hospital  is  facing  serious  challenges.  The  back-ups  power 
systems  have  failed  for  unknown  reasons.  Without  power,  or  working  back-up  systems, 
critical  patients  are  in  dire  need.  The  emergency  room  is  already  becoming  busy  with 
increased  accidents  due  to  the  icy  conditions,  and  vulnerable  patients  with  exposure  to  the 
cold  from  a  morning  with  no  heat.  The  head  of  the  hospital  contacts  the  municipal 
government,  and  asks  for  help  transferring  and  sheltering  patients. 

Wednesday  2:00  pm 

The  Gapville  municipal  government  asks  the  local  Fire  and  Police  departments  to  call  in 
all  available  personnel  to  deal  with  the  slippery  streets,  and  the  overflow  in  the  crippled 
hospital.  Mobilizing  the  municipal  forces  is  difficult  due  to  the  chaotic  state  on  the  roads. 
It  takes  several  hours  for  help  to  arrive  on  the  streets  and  in  the  hospital. 

Wednesday  6:00  pm 

Within  twelve  hours,  Gapville  Hydro  workers  have  managed  to  restore  the  power  to  a 
limited  section  of  the  city.  However,  hydro  workers  are  beginning  to  find  suspicious 
activity  not  indicative  of  a  power  failure.  Notably,  backup  systems  for  the  McLaughlin 
Memorial  Hospital  and  the  Gapville  Transit  system  are  still  not  operational.  These 
backup  systems  run  on  generators,  an  independent  power  supply,  and  should  technically 
not  be  affected  by  the  power  outage.  Gapville  Hydro  arrives  with  additional  generators  at 
the  Hospital,  and  attempt  to  hook-up  the  network.  Again,  efforts  fail,  power  cannot  be 
restored,  and  patients  must  be  evacuated. 

Facing  the  first  night  with  no  heat  or  light,  many  of  Gapville’s  local  community  networks 
begin  to  mobilize  their  own  resources,  walking  to  neighbours  who  may  be  vulnerable, 
and  opening  the  community  centers  and  churches,  offering  shelter  for  those  who  need  to 
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get  warm.  Fuel  is  an  issue,  for  building  generators,  and  the  portable  generators  being 
supplied  by  local  citizens. 

Municipal  road  crews  are  still  hard  at  work,  as  is  Gapville  Hydro,  clearing  roads  and 
repairing  damaged  power  lines.  EMS  is  now  also  working  with  health  care  workers  to 
move  patients  from  the  Hospital  to  the  local  community  center,  which  has  been  outfitted 
with  an  industrial  sized  generator  donated  from  the  local  lumber  yard.  There  are 
concerns  that  fuel  for  the  generator  will  only  last  one  night. 

Thursday  7:00  am 

The  municipal  government  contacts  the  Province,  asking  them  to  request  the  assistance  of 
the  Anned  Forces  to  provide  specialized  equipment  to  help  with  the  road  clearing  and 
continuation  of  vital  services.  The  mayor  also  asks  for  additional  fuel  and  generators  to 
be  brought  in.  The  military  agree  to  allocate  some  of  their  resources  to  Gapville,  and 
announce  they  will  be  arriving  in  two  days.  As  the  ice-storm  affected  most  of  the 
province  of  Anticipate,  the  military  has  several  sites  they  need  to  attend  to,  and  inter- 
provincial  travel  is  still  slow. 

Thursday  1:00  pm 

After  working  around  the  clock  for  over  3 1  hours,  Gapville  Hydro  workers  discover  the 
back-up  system  failures  at  the  Hospital  and  the  Transit  System  are  due  to  computer 
issues,  and  not  the  downed  power  lines.  Hydro  sends  their  computer  analysts  to  the 
McLaughlin  Memorial  Hospital  and  the  GapTransit  main  street  terminal,  to  try  to  reboot 
the  back-up  emergency  systems. 

Thursday  5:00  pm 

The  community  center  and  local  churches  are  becoming  crowded  with  people  seeking 
heat  and  light,  for  the  second  night  of  the  power  outage.  With  the  blackout  there  is 
limited  access  to  television  or  internet  for  infonnation.  Battery  operated  radios  are  used  to 
collect  any  information  available.  Due  to  the  blackout,  the  local  radio  station  is  still 
unable  to  broadcast  using  the  GapTV  tower,  so  the  only  news  available  is  from  the 
National  Radio  Service.  The  information  is  broad,  and  community  specific  information  is 
hard  to  come  by.  The  only  specific  information  for  Gapville  is  that  Hydro  Company  is 
still  unable  to  provide  a  time-frame  for  restoration  of  power. 

Thursday  10:00  pm 

Volunteers  and  Emergency  Response  agencies  are  doing  what  they  can  to  identify 
vulnerable  groups,  and  to  get  to  those  most  in  need  of  help.  The  senior’s  home  is  in  the 
blackout  area,  and  exposure  to  cold  is  becoming  an  increasing  concern.  Many  private 
groups  have  banded  together  to  help,  providing  fuel,  generators,  hand  crank  cell  phone 
chargers,  blankets  and  candles.  Several  people  have  been  found  using  incredibly 
dangerous  practices  for  heat,  for  example,  using  gas  powered  generators  and  BBQ’s 
indoors. 
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Friday  9:30  am 

Hydro  authorities  contact  Communications  Co.,  the  software  and  hardware  suppliers  for 
the  systems  at  the  hospital  and  the  transit  system.  Gapville  Hydro  explains  that  they  have 
been  unable  to  reboot  the  back-up  systems,  and  there  is  still  no  power  supply  at  the 
Hospital  or  at  the  Transit  System.  The  Communications  Company  sends  senior  analysts 
to  the  power  plant. 

Friday  12:00  pm 

Power  has  been  restored  at  GapTV,  where  the  city’s  communication  tower  resides. 
Broadcast  resumes  in  a  limited  for  local  television  and  radio.  Gapville  Hydro  expects  it 
will  take  another  two  days  for  power  to  fully  return  to  the  city.  The  lack  of  information  is 
beginning  to  cause  the  formation  of  rumours.  With  two  nights  of  no  heat  or  light,  citizens 
are  getting  frustrated,  and  are  wondering  when  the  power  will  be  back  online.  There  have 
been  rumours  of  a  fire  at  the  Hydro  Station  that  will  make  power  restoration  incredibly 
difficult,  and  could  take  up  to  a  month  to  fix.  Many  people  are  planning  larger  scale 
evacuations  due  to  this  misinformation,  some  are  planning  on  staying  with  family  in  other 
cities  and  many  are  hoping  that  commuter  trains,  or  flights  will  soon  be  running.  The 
train  station  and  airport  are  beginning  to  see  a  lot  of  people  arriving  frustrated  with  plans 
of  evacuation  and  no  way  to  carry  them  out. 

INJECT  -  Task  #1 

Saturday  8:00  am 

The  Armed  Forces  arrive  in  Gapville  with  road  clearing  materials,  supplies  for  shelters, 
generators  and  additional  fuel  supplies.  The  arrival  of  the  military  provides  some  relief  to 
the  distressed  citizens.  Gapvillians  feel  that  their  Mayor  has  taken  some  steps  to  ensure 
their  safety,  but  are  still  frustrated  with  the  lack  of  communication. 

Saturday  2:30  pm 

The  Communications  Company  discovers  the  source  of  the  computer  failure  was  due  to 
compromised  hardware.  The  Hospital’s  computers  and  backup  safety  systems  were  built 
using  counterfeit  microchips  which  allow  backdoor  access  to  hackers.  According  to  the 
serial  numbers,  the  compromised  hardware  arrived  in  a  large  shipment  last  year.  This 
revelation  also  takes  the  power  failure  from  an  accident  to  an  attack.  When  this 
revelation  is  made,  the  Reframe  Federal  Government  is  alerted,  and  investigators  from 
the  federal  cyber  task  force  are  sent  to  Gapville.  They  should  arrive  by  Monday  morning. 

Sunday  12:00  pm 

The  combined  efforts  of  fire  police  and  emergency  services,  working  alongside  the 
military,  volunteer  services,  municipal  crews,  and  Gapville  Hydro  has  made  serious 
headway  into  road  clearing  and  power  restoration.  Over  a  quarter  of  the  city  has  clear 
roadways  and  power.  Several  churches  and  public  buildings  can  now  be  safely  used  for 
shelter,  with  heat  and  light. 
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Monday  11:00  am 

Private  analysts  track  down  all  the  infected  computer  systems  in  the  city.  In  total,  three 
infected  systems  are  identified.  The  first  identified  is  at  the  McLaughlin  Memorial 
Hospital.  The  newly  upgraded  computer  patient  tracking  systems  are  compromised. 

These  systems  run  on  a  national  E-Health  network,  and  there  is  concern  that  confidential 
medical  information  all  across  the  country  of  Reframe  has  been  hacked.  The  local 
GapTransit  system  has  compromised  hardware  in  its  recently  installed  navigation 
systems.  And  finally,  the  National  Bank  switched  to  new  teller  and  ATM  software  6 
months  ago,  for  all  of  their  branches  across  the  province. 

Monday  2:00  pm 

The  potential  size  and  scope  of  the  computer  recall,  and  the  damage  expected  to  the  city's 
infrastructure  during  recalls  and  retrofitting  is  overwhelming.  The  obviously  malicious 
and  well-coordinated  computer  infiltration  has  federal  investigators  concerned  it  is  a 
terrorist  attack. 

The  freezing  rain  and  ice  pellets  end  and  the  skies  clear.  It  is  still  very  cold,  at  -22 
degrees  Celsius.  The  change  in  weather  allows  municipal  road  workers  and  the  military 
to  get  ahead  on  the  road  clearing.  Hydro  workers  are  also  able  to  get  almost  half  of  the 
city's  power  up  and  running. 

Monday  4:30  pm 

Communications  Co.  send  specialists  to  the  McLaughlin  Memorial  Hospital,  to  replace 
their  faulty  systems.  The  hospital  is  still  without  main  power,  but  once  the  backup 
computer  systems  are  restored,  the  generators  are  functioning,  the  ICU  and  Emergency 
Room  have  limited  power,  and  there  are  limited  lights,  but  there  is  heat.  It  will  still  take 
time,  however,  to  get  the  patient  tracking  and  E-Health  networks  secured. 

Tuesday  9:00  am 

The  Federal  Government,  the  Gapville  Municipal  government  and  P-RAM 
communications  hold  a  press  conference  at  Gap  TV  for  those  residents  with  power.  The 
Gapville  Mayor  explains  that  power  is  coming  back  on-line,  but  will  not  be  fully  restored 
for  two  more  days.  The  weather  forecasts  are  looking  clear  for  the  next  week,  and  should 
allow  the  road  crews  and  military  to  complete  their  work  on  clearing  the  city  within  the 
next  few  days  as  well.  The  mayor  provides  a  list  of  available  shelters,  and  some 
recommendations  for  health  and  safety  during  this  difficult  time.  The  federal  cyber  task 
force  investigators  then  explain  that  the  power  failures  are  not  fully  caused  by  the  ice 
storm.  There  were  computer  failures  at  the  Hospital,  the  GapTransit  System  and  the 
National  Bank,  caused  by  malicious  compromised  hardware.  The  investigators  explain 
that  embedded  chips  are  built  into  the  computers,  and  allow  access  to  vital  infrastructure 
computer  systems.  Because  of  these  hacked  computer  chips,  back-up  emergency  systems 
failed,  and  have  created  a  devastating  breach  of  privacy  in  health  and  financial  records, 
nationwide.  The  government  asks  people  to  remain  calm,  and  not  to  attempt  to  withdraw 
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their  savings  from  the  banks.  Officials  ask  for  patience,  and  trust  that  they  will  be  able  to 
correct  these  vital  systems  soon. 

Communications  Co.  explains  they  were  victims  of  counterfeit  hardware  purchase.  Due 
to  the  recall  of  these  systems,  the  banking  system  will  be  off-line  until  security  can  be 
ensured,  and  the  transit  system  will  not  be  operational  for  at  least  a  week.  The  hospital 
has  backup  systems  functioning,  and  full  power  will  be  restored  soon,  but  all  non- 
essential  procedures  will  be  suspended  until  the  E-Health  network  can  also  be  secured. 

Tuesday  2:00  pm 

Gapvillians  are  all  reeling  from  the  breadth  and  scope  of  the  cyber  attack  on  their  nation. 
The  entire  country  feels  the  effect  of  the  largest  national  bank  going  off-line,  and  the 
province  is  still  mostly  covered  in  ice.  The  official  press  conference  has  left  people 
feeling  vulnerable  and  scared.  There  is  a  run  on  the  banks,  with  many  people  attempting 
to  withdraw  their  savings.  But  with  systems  down,  the  tellers  are  unable  to  complete  any 
transactions.  As  people  become  more  frightened,  security  concerns  arise,  and  police  are 
called  away  from  their  efforts  in  the  hospitals  and  on  the  ice-clogged  streets  to  guard  the 
banks,  and  keep  some  order. 

Tuesday  6:00  pm 

The  city  is  preparing  for  another  night  with  no  heat  in  many  areas.  Roads  are  still  blocked 
in  many  smaller  streets,  but  the  main  thoroughfares  and  the  highway  have  been  cleared. 
While  the  municipal  road  crews  finish  the  clearing  the  military  turns  to  helping  with 
shelter  for  the  over  100,000  people  still  without  power.  At  the  same  time,  the  private 
sector  analysts  and  federal  investigators  are  working  hard  on  the  first  recalls,  and  tracking 
down  the  source  of  the  cyber-attack. 

Task  #2 
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M.6  Scenario  Assessment  Guide:  Scenario  Assessment 
Questionnaire  -  Blackout  Scenario 


This  form  is  designed  to  capture  a  reader’s  understanding  of  what  contributes  to  the  complexity  of  a  given 
scenario.  Potential  contributions  to  complexity  for  a  scenario  are  described  below  in  terms  of  impact, 
uncertainty  and  vulnerability,  with  specific  challenges  or  areas  noted  for  each.  After  reading  the  scenario, 
the  reviewer  is  asked  to  rate  the  degree  to  which  each  challenge  was  evident  in  the  scenario.  The  results 
will  be  compared  with  data  collected  from  other  reviewers  to  select  a  suitably  complex  scenario  for  use  in  a 
simulation  exercise. 

Q1  Overall,  which  of  the  following  descriptors  best  describes  the  scenario? 

□  Simple 

□  Complicated 

□  Complex 


Please  rate  the  extent  to  which  each  of  the  following  challenges  is  evident  in  the  scenario.  Use  the  scale 
ranging  from  0  (not  at  all  evident)  to  4  (strongly  evident). 


0 

i 

2 

3 

4 

Impact 

Q2 

Impact  severity 

(i.e.,  number  of  fatalities  /casualties, 
amount  of  physical  damage/  psychological 
trauma,  degree  of  social  disruption/multiple 
events) 

Q3 

Impact  scope 

(i.e.,  breadth  of  impact  on  people,  services, 
structures,  societal  functions ) 

Q4 

Impact  timing 

(i.e.,  no  threat/warning  phase,  sudden 
onset,  sustained  duration,  and/or  recurring 
event ) 

Q5 

Media  involvement 

(i.e.,  extensive  media  coverage,  lack  of 
access  to  official  sources,  inappropriate 
use  of  social  media  tools) 

Q6 

Political  processes 

(i.e.,  multiple  organizations  involved, 
multiple  jurisdictions,  and/or  multiple  levels 
of  government,  public  fear  and/or  mistrust 
in  officials) 

Uncertainty 
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Q7 

Novelty  of  situation 

(i.e.,  novel  hazard,  malicious  cause,  lack  of 
experience,  lack  of  controllability,  and/or 
latent  impacts) 

Q8 

Anticipation  and  planning 

(i.e.,  lack  of  planning,  rehearsal,  and 
sharing  of  plans,  and/or  lack  of  integration 
of  plans) 

Q9 

Data  and  information 

(i.e.,  lack  of  access  to  information, 
inaccurate  information,  lack  of  credible 
information  sources,  excessive  volume  of 
data,  and/or  fast  speed  of  information  flow) 

Q10 

New  organizations  and  partners 

(i.e.,  presence  of  non-traditional 
organizations  and  partners,  and/or 
presence  of  emergent  organizations  or  ad- 
hoc  groups  that  seif-organize  to  fiii 
perceived  resource  gaps) 

Q11 

Changing  context 

(i.e.,  big  changes  in  situation  context,  fast 
speed  of  change,  actual/perceived 
negative  consequences  of  change,  and/or 
negative  impacts  of  intervention  (negative 
secondary  effects) 

Q12 

Flexibility  of  interpretive  frameworks 

(i.e.,  imposed/unilateral  processes,  lack  of 
common/shared  processes,  and/or  lack  of 
creativity/improvisation) 

Vulnerability 

Q13 

Economic  development 

(i.e.,  lack  of  resources,  lack  of  economic 
diversity,  unequal  distribution  of  wealth, 
and/or  economic  instability) 

Q14 

Social  capital 

(i.e.,  large  number  of  at-risk  populations, 
lack  of  social  support,  poor  citizen 
participation  in  volunteering  and  decision 
making,  poor  sense  of  community,  and/or 
excessive/insufficient  attachment  to  place) 

Q15 

Community  competence 

(i.e.,  political  instability,  poor  leadership 
skills,  low  literacy  and  education  levels, 
lack  of  experience  with  similar  hazards, 
and/or  lack  of  community  partnerships  and 
support) 
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Q16 

Information  and  communication 

(i.e.,  unreliable  or  excessive  amounts  of 
extraneous  information,  public  opinion 
ignored,  lack  of  trusted  media 
spokesperson,  ineffective  risk  messaging, 
and/or  poor  communication  infrastructure 
capabilities) 

Q17 

Other  infrastructure 

(i.e.,  high  degree  of  interdependence 
among  critical  systems,  lack  of  redundant 
or  backup  systems,  and/or  weak  or 
blocked  distribution  networks ) 

Q18  In  your  estimation,  which  of  the  following  factors  contributed  most  to  the  level  of  complexity  in  the 
scenario?  Please  check  one  box  and  explain  your  choice. 

□  Impact 

□  Uncertainty 

□  Vulnerability 

□  All  of  the  above  (factors  influenced  the  assessment  equally) 

Please  explain  why: 


Please  rate  the  overall  level  of  complexity  of  the  scenario  on  a  scale  of  0  (not  at  all  complex)  to  4  (very 
complex). 


0 

1 

2 

3 

4 

Q19 

Overall,  1  would  describe  the  level  of 
complexity  for  the  scenario  as: 

Please  rate  your  level  of  agreement  with  the  following  statements  on  a  scale  of  0  (strongly  disagree)  to  4 
(strongly  agree).  _ _ _ _ 


0 

1 

2 

3 

4 

Q20 

The  first  part  of  the  scenario  is  realistic  / 
plausible. 

Q21 

The  second  part  of  the  scenario  is  realistic 
/  plausible. 

Q22 

The  timeline  for  the  scenario  is  realistic  / 
plausible. 
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M.7  Scenario  Assessment  Guide:  Mail  Scenario 


Monday  4:00  pm 

There  is  a  small  explosion  and  fire  at  the  Sort  &  Send  Mail  Delivery  (SSMD)  warehouse 
in  Gapville.  The  employees  put  out  the  fire  with  a  couple  of  fire  extinguishers  while 
calling  911.  There  are  approximately  50  other  employees  in  the  building. 

Three  employees  are  injured  from  the  event.  The  employee  who  was  handling  the 
package  at  the  time  of  explosion  has  more  serious  injuries.  The  two  employees  with 
minor  injuries  are  given  first  aid  and  taken  by  another  employee  to  the  hospital.  They 
don’t  want  to  wait  for  the  ambulance  and  don’t  feel  the  need  for  emergency  treatment. 
They  are  seen  for  workers’  compensation  reasons  and  released  from  the  hospital.  All  of 
the  remaining  employees  in  the  facility  stay  at  the  warehouse  and  continue  working. 

The  Gapville  fire  department  arrives  and  gives  initial  medical  care  to  the  more  seriously 
injured  employee.  The  paramedics  then  arrive  to  take  over  his  care.  They  transport  him  to 
the  hospital  trauma  centre  where  he  is  seen  in  the  emergency  department  and,  given  his 
injuries,  is  admitted  to  the  hospital  for  treatment  of  burns  and  a  fractured  arm.  The 
ambulance  crew  returns  to  service  and  responds  to  other  calls  throughout  the  city.  The 
fire  department  ensures  that  there  is  no  further  damage  or  fire  and  calls  the  Gapville 
police  department  since  it  appears  to  be  a  suspicious  fire.  The  fire  crews  return  to  their 
stations.  The  police  determine  from  SSMD  records  that  the  package  had  been  dropped 
off  early  in  the  day  and  was  to  be  delivered  to  the  Gap  TV  television  station  the  next  day. 

Monday  7:00  pm 

All  of  the  emergency  personnel  in  Gapville  -  EMS,  Fire,  Police  and  Hospital  staff 
complete  shift  changes  between  4:00  pm  and  6:00  pm.  Those  responders  who  were  on  the 
scene  of  the  SSMD  explosion  now  are  off-duty,  and  travel  to  home,  schools,  restaurants 
and  other  recreational  activities. 

Gapville  Police  Station  #2  receives  an  anonymous  call,  announcing  that  a  dirty  bomb  has 
been  sent  to  the  GapTV  television  station.  The  police  connect  the  call  with  the  situation  at 
the  SSMD  warehouse  and  call  the  HazMat  team,  Federal  Investigators,  the  Military,  and 
the  Provincial  Government.  If  this  caller’s  information  is  true,  this  revelation  takes  the 
SSDM  explosion  from  a  potential  accident  to  a  radiological  attack.  The  military  announce 
they  will  arrive  within  72  hours. 

Monday  10:00  pm 

After  a  brief  investigation,  the  HazMat  team  detennines  that  the  explosion  spread 
approximately  one  ounce  of  Cesium  137  in  the  building.  Initial  readings  of  the  exposure 
level  at  the  site  were  100  rems/hour;  however,  those  who  were  one  meter  from  the 
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explosion  received  a  dose  of  1000  rems/hr  and  those  10  meters  away,  between  100  and 
543  rems/hr.  Those  who  handled  the  patient  were  exposed  to  slightly  higher  dosages 
because  the  dust  came  off  of  the  patient’s  clothing  and  transferred  to  them.  Other  hospital 
staff  and  patients  in  the  treatment  area  of  the  emergency  room  received  between  10-15 
rems/hour. 

The  explosion  was  contained  for  the  most  part  in  the  SSDM  facility  and  all  those  who 
had  been  there  were  exposed  and  contaminated  to  varying  degrees.  Employees  had  used 
fire  extinguishers  to  combat  the  flames,  and  the  force  of  extinguishers  spread  the  Cesium 
137  around  the  ventilation  ducts  in  the  warehouse.  All  responders  were  also  exposed  and 
some  experience  some  degree  of  radiation  sickness  over  the  coming  days  depending  on 
how  close  they  were  to  the  source  and  the  patient. 

In  addition,  for  those  in  the  facility,  the  long  tenn  risks  of  cancer  increased  significantly 
as  a  result  of  exposure  to  inhalation  of  contaminated  dust  particles.  Employees,  police, 
EMS  and  fire  fighters  received  a  dose  of  radiation  that  increased  their  risk  of  cancer  to  1 
in  5.  Those  in  the  hospital  emergency  room  increased  their  cancer  risk  to  1  in  50.  An 
additional  risk  was  created  by  the  dust  particles  on  the  clothing  and  equipment  that  was  in 
taken  home,  or  into  the  public,  by  individuals.  This  increased  the  cancer  risk  in  areas  of 
the  public  to  about  1  in  100. 

At  about  the  same  time  that  the  HazMat  team  confirmed  the  presence  of  Cesium  137,  the 
SSMD  employee  who  was  hospitalized  earlier  begins  to  show  initial  symptoms  of 
radiation  exposure  and  his  condition  becomes  critical.  When  notified  by  the  police,  the 
hospital  calls  a  “code  brown”,  for  hazardous  materials,  and  closes  the  emergency 
department  to  all  incoming  cases.  After  three  hours,  all  three  injured  SSMD  employees 
begin  to  show  signs  of  radiation  sickness.  The  most  seriously  injured  patient  is  now 
critically  ill  while  the  others,  who  are  at  home,  are  less  ill  but  are  showing  signs  of  nausea 
and  red  skin. 

Tuesday  9:00  am 

Tracking  is  underway  for  all  those  who  were  in  contact  with  the  SSMD  patients  or  who 
were  at  the  scene  of  the  event.  In  addition,  Public  Health  authorities  are  tracking  any 
people  who  were  at  the  ER  when  the  SSMD  patients  came  in.  Working  in  conjunction 
with  Gap  TV,  the  authorities  hold  a  press  conference.  They  ask  that  anyone  who  was  in 
the  exposed  buildings,  or  near  an  affected  victim  to  go  to  the  hospital  to  get  checked  out. 
The  municipal  authorities  have  set  up  temporary  exam  centers  in  the  parking  lot  of  the 
McLaughlin  Memorial  Hospital.  All  who  were  directly  exposed  will  become  ill  within  72 
hours  and  will  be  showing  different  signs  of  radiation  exposure  over  the  coming  weeks. 

Tuesday  7:30  pm 

Hospital  staff  has  worked  overtime,  dealing  with  the  influx  of  patients,  coupled  with  the 
loss  of  employees  who  have  been  exposed  to  radiation. 
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Hospital  authorities  request  help  from  the  neighbouring  city’s  Hospital.  There  is 
confirmation  that  the  other  SSMD  employees,  the  paramedics  and  the  initial  ER  staff  who 
treated  the  SSMD  employees  were  exposed  to  radiation  and  with  treatment  will  survive, 
but  with  unknown  impacts.  All  of  the  fire  fighters,  EMS  and  police  at  the  scene  were 
also  exposed  to  varying  degrees  of  radiation.  At  the  moment,  22  people  are  hospitalized 
for  treatment  of  sub-lethal  but  serious  effects  of  radiation  exposure;  another  65  are 
following  out-patient  treatments.  Thousands  have  arrived  at  the  examination  centers  set 
up  in  the  hospital  parking  lot.  No  one  can  estimate  however,  the  degree  of  exposure  and 
the  risk  for  each,  either  in  the  short  or  long  term.  Family  members  for  most  of  the 
responders  have  been  found  to  have  varying  levels  of  exposure  as  a  result  of  the 
contaminated  clothing  brought  home.  Potentially  exposed  children  are  now  the  focus  of 
intensive  surveillance  and  monitoring.  It  is  estimated  that  the  risk  for  cancer  may  be  as 
high  as  1  in  50  for  some  of  them. 

Wednesday  9:00am 

GapTV  broadcasts  another  press  conference.  Authorities  announce  there  have  been  no 
further  bombs  or  threats.  Sadly,  there  is  one  confirmed  death,  the  critically  injured  SSMD 
employee,  as  a  result  of  acute  radiation  exposure.  Due  to  the  amount  of  contamination, 
the  employee  was  buried  in  a  lead  coffin  sealed  in  concrete.  It  is  likely  that  the  two 
SSMD  employees  who  initially  exhibited  minor  physical  injuries  will  also  pass  away. 
Federal  investigators  have  arrived  and  announce  they  have  yet  to  track  down  the  source 
or  motives  for  this  attack.  The  public  is  scared  and  uncertain  as  to  the  source  of  the 
radiological  attack,  and  are  very  concerned  about  their  exposure  and  their  children’s 
exposure  to  radiation.  Equally  there  is  a  loss  of  public  trust.  How  could  authorities  have 
allowed  contaminated  responders  to  go  out  into  the  public  after  their  shift?  Why  was  this 
not  caught  in  time? 

Officials  are  unable  to  give  full  exposure  estimates  but  have  taken  a  variety  of  actions 
intended  to  restore  public  confidence.  The  fire  stations  that  went  to  the  fire  have  been 
closed  and  their  equipment  and  vehicles  are  being  decontaminated.  The  ambulance,  the 
hospital  emergency  room  (ER)  and  the  rooms  where  the  patients  were  handled  are  also 
being  decontaminated.  The  hospital  itself  has  completely  shut  down  and  moved  patients 
to  other  facilities. 

Wednesday  5:30pm 

The  police  investigation  identifies  those  responsible  and  where  they  assembled  the  bomb. 
They  have  so  far  eluded  the  police,  and  it  is  thought  they  may  have  fled  the  country. 

There  is  concern  that  they  possibly  contaminated  themselves  during  the  bomb  assembly, 
and  that  they  may  be  spreading  the  contamination  as  they  travel.  The  authorities  have  not 
been  able  to  detennine  how  they  left  or  where  they  went. 

The  perpetrators  used  a  suburban  house,  which  is  now  contaminated.  The  area  for  several 
blocks  around  it  is  evacuated  and  sealed  off  while  authorities  start  to  conduct  additional 
tests.  As  they  conduct  these  tests,  they  discover  a  natural  source  of  radon  which  creates 
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an  additional  concern  for  the  people  living  in  that  area.  There  is  a  lot  of  confusion  about 
different  sources  and  the  public  is  not  distinguishing  between  types  of  radiation  but  are 
insisting  that  they  be  protected  from  any  type. 

Thursday  8:00am 

There  have  been  no  further  bombs  or  threats;  however,  there  has  been  additional  loss  of 
life.  Over  15  people  have  died.  Local  funeral  homes  have  stated  that  they  do  not  want  to 
handle  the  bodies  for  fear  of  contamination.  The  families  are  upset,  believing  they  are  not 
receiving  adequate  infonnation  or  support.  There  is  inconsistent  media  coverage  with 
different  “experts”  recommending  a  wide  variety  of  methods  for  handling  the  sick  and 
deceased,  including  burial  in  lead  coffins. 

Thursday  3:30pm 

The  military  arrive,  and  help  with  many  different  needs,  moving  critical  patients  from  the 
shopping  mall  to  more  appropriate  treatment  centers,  such  as  the  walk-in  clinics  and  the 
assisted  living  homes.  They  also  set  up  additional  evacuation  centers,  for  those  displaced 
from  the  affected  area,  and  set  up  decontamination  sites  at  many  different  locations 
across  the  city. 

Thursday  5:00pm 

Family  members  for  all  of  the  responders  are  found  to  have  varying  levels  of  exposure  as 
a  result  of  the  contaminated  clothing  being  brought  home.  Children  are  the  focus  of 
intensive  surveillance  and  monitoring.  Media  reports  have  suggested  that  they  will  have  a 
long-tenn  increased  risk  of  developing  cancer,  which  creates  some  public  anxiety  and 
demand  for  more  testing  and  information.  There  is  significant  internet  chatter  regarding 
the  long-term  outcomes  of  this  event. 

Task  #1 

Friday  10:15  am 

Each  day,  people  turn  up  at  the  hospital  asking  to  be  screened  and  treated,  insisting  that 
they  have  been  exposed  or  manifesting  symptoms  of  radiation  exposure.  Additionally, 
people  are  demanding  to  know  how  long-term  monitoring  will  be  carried  out,  and  in  what 
way  will  they  be  treated  and/or  compensated  if  they  develop  cancer. 

There  is  considerable  scepticism  that  the  estimate  of  those  exposed  is  accurate;  many 
suspect  the  number  is  being  downplayed  by  authorities.  Thus,  there  is  a  growing  demand 
that  Prussian  Blue,  the  only  known  treatment,  to  be  widely  distributed  to  the  population, 
even  to  anyone  who  might  have  been  remotely  exposed  to  the  contamination.  Prussian 
Blue  is  not  licensed  for  use  in  this  country  and  requires  a  special  license  from  Public 
Health  for  this  purpose.  Officials  are  reluctant  to  provide  anything  other  than  limited 
access.  There  is  not  enough  Prussian  Blue  available  to  provide  widespread  treatment,  so 
only  those  first  responders  with  confirmed  exposure  have  been  given  the  treatment.  The 
first  responders  demand  that  their  family  members  be  provided  access  also.  The  SSMD 
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employees,  who  suffered  just  as  much  radiation  exposure  as  the  first  responders,  demand 
treatment  both  for  themselves  and  for  their  families. 

Saturday,  12:00pm 

Monitoring  of  people  living  in  the  neighbourhood  where  the  bomb  assembly  took  place 
has  not  found  any  people  with  exposure  to  radiation.  Officials,  however,  are  not  able  to 
reassure  people  that  there  will  be  no  long-tenn  negative  effects.  Those  who  have  been 
evacuated  are  demanding  assistance  in  covering  their  costs,  as  well  as  compensation  for 
their  homes  which  they  say  will  not  sell  any  time  in  the  foreseeable  future. 

The  SSMD  site  has  been  sealed  for  both  decontamination  and  the  police  investigation. 
Other  businesses  in  the  area  have  shut  down,  and  tests  are  being  conducted  to  detennine 
if  there  has  been  any  leakage  from  the  SSMD  facility.  Traffic  in  the  area  remains  closed 
which  creates  a  number  of  problems  for  businesses  in  the  vicinity.  Again,  military 
support  is  needed  to  handle  the  flow  of  traffic,  enforce  the  evacuation  zone,  and  provide 
shelter  and  support  for  those  displaced  persons.  The  military  work  with  local  authorities, 
and  volunteer  agencies  to  try  to  mitigate  some  of  the  more  unpleasant  effects  of 
evacuation  and  decontamination. 

Monday  9:00am 

It  is  announced  that  the  SSMD  facility  will  have  to  be  demolished  along  with  all  of  the 
material  that  was  in  it.  All  of  the  packages  and  documents  in  the  facility  have  to  be 
destroyed.  This  creates  problems  for  some  of  the  companies  affected.  Insurance  claims 
pour  in,  and  SSMD  is  struggling  to  maintain  its  customer  base  across  the  country.  Courier 
services  in  general  have  seen  a  significant  drop  in  business. 

Announced  plans  to  re-open  the  fire  station  and  the  hospital  are  met  with  strong  public 
opposition.  The  public  demands  assurance  that  the  decontamination  efforts  will  be  100% 
effective  and  that  there  is  no  further  risk  of  exposure. 

Thursday  9:00am 

There  are  a  number  of  lawsuits  being  filed  against  the  emergency  services  for  not 
protecting  the  public  and  further  endangering  people  in  the  vicinity  of  the  station  and 
along  the  routes  that  they  traveled  on.  The  hospital  has  a  series  of  legal  actions  claiming 
that  people  died  as  a  result  of  the  decision  to  close  and  transfer  them. 

There  are  numerous  issues  related  to  the  removal  of  contaminated  material  -  i.e.,  where  it 
will  go  and  how  it  will  be  disposed  of.  This  has  caught  the  media’s  attention  as  well  as 
that  of  numerous  groups  who  do  not  want  it  transported  along  city  streets  to  wherever  it 
will  finally  end  up.  It  has  been  estimated  that  the  main  site  clean-up  will  take  6  to  8 
months.  There  are  conflicting  reports  about  the  risks  posed  to  the  community  during  this 
activity.  There  is  also  a  debate  about  how  much  of  an  area  should  be  cleaned  up  -  2 
blocks  or  10  blocks  around  each  of  the  different  sites.  The  amount  of  waste  from  the 
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SSMD  warehouse  is  estimated  to  fill  a  football  field  waist  deep.  There  is  a  suggestion 
that  the  entire  area  be  abandoned  and  closed  off  for  20  to  30  years. 

Clean-up  costs  are  estimated  to  be  in  the  billions  of  dollars.  There  is  an  ongoing  issue 
about  which  level  of  government  will  be  responsible  for  what  costs.  Insurance  companies 
are  claiming  that  since  it  was  an  act  of  terrorism,  any  policies  are  null  and  void. 

Task  #2 
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M.8  Scenario  Assessment  Guide:  Scenario  Assessment 
Questionnaire  -  Mail  Scenario 


This  form  is  designed  to  capture  a  reader’s  understanding  of  what  contributes  to  the  complexity  of  a  given 
scenario.  Potential  contributions  to  complexity  for  a  scenario  are  described  below  in  terms  of  impact, 
uncertainty  and  vulnerability,  with  specific  challenges  or  areas  noted  for  each.  After  reading  the  scenario, 
the  reviewer  is  asked  to  rate  the  degree  to  which  each  challenge  was  evident  in  the  scenario.  The  results 
will  be  compared  with  data  collected  from  other  reviewers  to  select  a  suitably  complex  scenario  for  use  in  a 
simulation  exercise. 

Q1  Overall,  which  of  the  following  descriptors  best  describes  the  scenario? 

□  Simple 

□  Complicated 

□  Complex 


Please  rate  the  extent  to  which  each  of  the  following  challenges  is  evident  in  the  scenario.  Use  the  scale 
ranging  from  0  (not  at  all  evident)  to  4  (strongly  evident). 


0 

i 

2 

3 

4 

Impact 

Q2 

Impact  severity 

(i.e.,  number  of  fatalities  /casualties, 
amount  of  physical  damage/  psychological 
trauma,  degree  of  social  disruption/multiple 
events) 

Q3 

Impact  scope 

(i.e.,  breadth  of  impact  on  people,  services, 
structures,  societal  functions ) 

Q4 

Impact  timing 

(i.e.,  no  threat/warning  phase,  sudden 
onset,  sustained  duration,  and/or  recurring 
event ) 

Q5 

Media  involvement 

(i.e.,  extensive  media  coverage,  lack  of 
access  to  official  sources,  inappropriate 
use  of  social  media  tools) 

Q6 

Political  processes 

(i.e.,  multiple  organizations  involved, 
multiple  jurisdictions,  and/or  multiple  levels 
of  government,  public  fear  and/or  mistrust 
in  officials) 

Uncertainty 
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0 

1 

2 

3 

4 

Q7 

Novelty  of  situation 

(i.e.,  novel  hazard,  malicious  cause,  lack  of 
experience,  lack  of  controllability,  and/or 
latent  impacts) 

Q8 

Anticipation  and  planning 

(i.e.,  lack  of  planning,  rehearsal,  and 
sharing  of  plans,  and/or  lack  of  integration 
of  plans) 

Q9 

Data  and  information 

(i.e.,  lack  of  access  to  information, 
inaccurate  information,  lack  of  credible 
information  sources,  excessive  volume  of 
data,  and/or  fast  speed  of  information  flow) 

Q10 

New  organizations  and  partners 

(i.e.,  presence  of  non-traditional 
organizations  and  partners,  and/or 
presence  of  emergent  organizations  or  ad- 
hoc  groups  that  seif-organize  to  fiii 
perceived  resource  gaps) 

Q11 

Changing  context 

(i.e.,  big  changes  in  situation  context,  fast 
speed  of  change,  actual/perceived 
negative  consequences  of  change,  and/or 
negative  impacts  of  intervention  (negative 
secondary  effects) 

Q12 

Flexibility  of  interpretive  frameworks 

(i.e.,  imposed/unilateral  processes,  lack  of 
common/shared  processes,  and/or  lack  of 
creativity/improvisation) 

Vulnerability 

Q13 

Economic  development 

(i.e.,  lack  of  resources,  lack  of  economic 
diversity,  unequal  distribution  of  wealth, 
and/or  economic  instability) 

Q14 

Social  capital 

(i.e.,  large  number  of  at-risk  populations, 
lack  of  social  support,  poor  citizen 
participation  in  volunteering  and  decision 
making,  poor  sense  of  community,  and/or 
excessive/insufficient  attachment  to  place) 

Q15 

Community  competence 

(i.e.,  political  instability,  poor  leadership 
skills,  low  literacy  and  education  levels, 
lack  of  experience  with  similar  hazards, 
and/or  lack  of  community  partnerships  and 
support) 
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0 

1 

2 

3 

4 

Q16 

Information  and  communication 

(i.e.,  unreliable  or  excessive  amounts  of 
extraneous  information,  public  opinion 
ignored,  lack  of  trusted  media 
spokesperson,  ineffective  risk  messaging, 
and/or  poor  communication  infrastructure 
capabilities) 

Q17 

Other  infrastructure 

(i.e.,  high  degree  of  interdependence 
among  critical  systems,  lack  of  redundant 
or  backup  systems,  and/or  weak  or  blocked 
distribution  networks ) 

Q18  In  your  estimation,  which  of  the  following  factors  contributed  most  to  the  level  of  complexity  in  the 
scenario?  Please  check  one  box  and  explain  your  choice. 

□  Impact 

□  Uncertainty 

□  Vulnerability 

□  All  of  the  above  (factors  influenced  the  assessment  equally) 

Please  explain  why: 


Please  rate  the  overall  level  of  complexity  of  the  scenario  on  a  scale  of  0  (not  at  all  complex)  to  4  (very 
complex). 


0 

1 

2 

3 

4 

Q19 

Overall,  1  would  describe  the  level  of 
complexity  for  the  scenario  as: 

Please  rate  your  level  of  agreement  with  the  following  statements  on  a  scale  of  0  (strongly  disagree)  to  4 
(strongly  agree).  _ _ _ _ 


0 

1 

2 

3 

4 

Q20 

The  first  part  of  the  scenario  is  realistic  / 
plausible. 

Q21 

The  second  part  of  the  scenario  is  realistic 
/  plausible. 

Q22 

The  timeline  for  the  scenario  is  realistic  / 
plausible. 
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M.9  Scenario  Assessment  Guide:  Train  Derailment  Scenario 


Monday  11:45  am 

It  is  the  first  Monday  in  June,  during  a  heat  wave  that  has  left  Gapville  hot  and  humid. 
At  1 1 :45  am  a  freight  train  derails  only  half  a  kilometre  away  from  the  National  Rail 
(NR)  station.  The  NR  radio  from  the  station  for  help,  and  inform  responders  there  was  a 
train  derailment,  and  an  explosion  has  occurred. 

Within  minutes,  first  responders  are  on  the  scene.  Six  chemical  containment  cars  are 
breached,  sending  a  large  yellowish  haze  of  chemical  up  into  the  air.  Paramedics  wear 
respirators  and  gloves  as  they  scramble  to  get  the  seriously  wounded  and  chemically 
exposed  to  hospital. 

Monday  1:00  pm 

NR  authorities  have  identified  the  chemical  leak  as  Carbonyl  Chloride,  a  phosgene 
chiefly  used  in  the  production  of  polymers  including  polyurethanes,  and  polycarbonates. 
The  toxicity  of  a  phosgene  is  mainly  due  to  the  HC1  (hydrogen  chloride)  that  is  released 
with  its  reaction  with  water.  Even  small  amounts  of  water  can  trigger  the  toxic  reaction, 
such  as  clouds,  or  even  entry  into  the  lungs. 

A  fatal  dose  of  phosgene  eventually  leads  to  shallow  breathing  and  retching,  rapid  pulse, 
an  ashen  face  and  the  discharge  of  2  liters  of  yellow  liquid  from  the  lungs  each  hour  for 
the  48  hours  of  the  drowning  spasms  that  occur  following  exposure. 

The  protocol  for  an  HC1  leak  is  immediate  removal  from  the  area,  and  protection  for 
those  downwind  of  the  chemical.  Equally  important,  those  who  have  been  in  direct 
contact  with  the  chemical  should  not  be  taken  to  hospital,  as  they  carry  contaminants  on 
their  clothing  and  skin.  The  hospital  is  immediately  notified  of  the  nature  of  the 
chemical  contamination.  The  hospital  immediately  performs  decontamination  of  all  the 
train  derailment  victims  and  health  care  workers  who  came  in  contact  with  these 
patients. 

Monday  4:00  pm 

Within  a  couple  of  hours,  the  chemical  cloud  has  spread  from  the  accident  site  at  the 
central  north  end  of  the  city,  through  the  downtown  corridor  to  south  central  Gapville, 
effectively  splitting  the  city  into  two  halves.  Due  to  the  high  level  of  humidity  the 
normally  heavy  gas  is  able  to  react  with  low  laying  haze,  and  slowly  drift  through  the 
city. 

Authorities  are  recommending  evacuation  for  Gapvillians  in  the  directly  affected  areas 
due  to  the  highly  toxic  nature  of  Carbonyl  Chloride,  and  the  forecasted  continued  hot 
and  humid  weather  conditions. 
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Monday  5:00  pm 

With  the  size  and  scope  of  the  evacuation  and  decontamination,  Gapville’s  resources  are 
stretched  to  the  limit.  Police  and  the  Fire  department  are  still  on  scene  at  the  site  of  the 
derailment.  Rescue  and  clean-up  are  incredibly  difficult  with  the  chemical  cloud. 

The  McLaughlin  Memorial  Hospital  is  within  the  evacuation  zone,  and  requires  extra 
help  in  moving  patients  to  the  High  School  and  the  Shopping  Mall.  EMS  is  working 
with  the  municipal  government,  volunteer  agencies  and  the  hospital  to  evacuate  to 
safety. 

The  municipal  authorities  contact  the  Province,  who  immediately  sends  additional 
resources  to  set  up  temporary  decontamination  sites  as  well  as  police  officers  to  begin 
enforcing  an  evacuation  zone.  The  province  also  contacts  the  National  military,  who 
announce  they  will  arrive  on  Thursday  morning.  Some  resources  may  be  there  as  early 
as  Wednesday,  including  temporary  shelters,  decontamination  supplies  and  personnel  to 
assist. 

Monday  6:00  pm 

The  municipal  government  creates  an  evacuation  message,  however,  the  GapTV  studios, 
which  contain  the  local  broadcast  tower,  are  also  inside  the  evacuation  zone,  making 
local  radio  and  television  transmissions  impossible.  A  remote  news  van  is  used  to 
videotape  the  information,  and  send  it  to  the  national  networks.  The  internet  is  rapidly 
becoming  the  most  used  source  for  local  information. 

The  local  authorities  seek  to  calm  the  people,  explaining  that  contamination  through  the 
air  is  far  less  toxic  than  direct  contact.  Equally  they  prescribe  home  decontamination 
procedures. 

“At  temperatures  above  7.6°  C  it  is  most  likely  that  exposure  will  be  solely  to  phosgene 
vapor,  a  more  harmless  exposure  than  direct  contact  with  the  chemical.  If  there  is  no 
evidence  of  exposure  to  liquid  phosgene,  decontamination  simply  requires  removing  the 
victim  from  the  contaminated  area  and  removing  outer  clothing  which  should  be  bagged 
and  disposed  of  Soapy  water  should  be  used  to  clean  the  skin  using  a  rinse,  towel  off, 
repeat  process  if  liquid  has  been  observed.  Hair  should  be  thoroughly  cleaned  using 
soap  and  water,  with  care  being  taken  to  prevent  wash  water  from  contacting  eyes.  ” 

The  local  authorities  also  outline  the  evacuation  zone,  an  1 1  kilometer  radius  around  the 
toxic  cloud.  This  takes  up  over  one  third  of  the  city,  a  devastatingly  large  area,  which 
includes  many  of  the  city’s  key  infrastructures  such  as  public  transit  and  the  bank. 

Monday  9:30  pm 

Evacuations  are  still  moving  slowly  and  mostly  on  foot.  The  toxic  cloud  ran  very  close 
along  the  entire  north-south  GapTransit  monorail  line,  and  the  Main  Street  terminal, 
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effectively  shutting  down  public  transportation.  Authorities  need  to  wear  respirators  and 
gown  and  gloves,  as  they  go  door  to  door  asking  residents  to  leave  the  affected  area. 

Tuesday  4:30  am 

The  evacuation  zone  is  declared  officially  cleared  of  residents.  Those  most  seriously 
affected  were  carried  on  stretchers  to  decontamination  and  treatment  centers.  Exertion 
increases  the  reaction  within  the  lungs,  so  even  ambulatory  patients  need  to  be  carried  if 
possible.  With  the  more  urgent  message  being  delivered  from  the  National  authorities, 
many  of  those  who  could  have  performed  a  home  decontamination  arrive  at  the 
treatment  sites. 

Tuesday  9:00  am 

Almost  a  day  later  at  the  site  of  the  train  derailment,  rescue  workers  have  retrieved  the 
last  of  the  victims.  It  is  becoming  obvious  that  the  leaking  chemical  cars  have 
contaminated  the  river,  which  runs  beside  the  tracks.  The  river  provides  the  city’s  local 
water  supply.  It  also  is  the  watershed  for  a  large  portion  of  the  province.  The  federal 
government  is  contacted,  and  they  send  in  environmental  experts  to  assess  the  damage 
being  done  to  the  city  and  province's  water  supply,  and  air  quality. 

Tuesday  9:15  am 

National  and  provincial  authorities  prepare  a  press  release  of  their  own.  A  state  of 
emergency  is  declared  for  the  province. 

“For  those  responders  in  the  area,  complete  chemical  protective  ensembles  should  be 
worn  to  ensure  protection.  Self-contained  breathing  apparatus  is  required  for  all  on-site 
personnel.  Volunteers  should  not  enter  the  evacuation  zone.  All  residents  in  the  affected 
zone  are  to  evacuate  uphill  and  upwind  without  moving  through  the  agent  cloud.  A  list 
of  shelters  and  decontamination  /  treatment  sites  are  available.  It  is  recommended  you 
seek  treatment  if  you  experience  shortness  of  breath,  or  a  burning  of  the  eyes  or  mouth. 
Water  contaminated  by  HCl  poses  potential  hazards,  and  clean  up  of  this  spill  may  be 
extensive.  Do  not  drink  any  tap  water.  Use  bottled  water  only.  ” 

INJECT  -  Task  #1 

Wednesday  10:00  am 

Evacuation  centers  are  full,  and  the  military  has  arrived  to  help  with  the  displaced 
individuals,  setting  up  shelters,  medical  stations,  and  distributing  food  and  water 
supplied  through  volunteer  organizations  throughout  the  province. 

The  federal  environmental  team  conducted  preliminary  analysis,  and  are  disheartened 
with  the  results.  With  the  contents  of  all  six  cars  drained  into  the  water  supply,  and  the 
toxic  cloud  hovering  above  the  city’  downtown  core,  water,  land  and  air  are  all  affected. 
With  two  days  of  exposure,  there  are  many  surfaces  in  the  evacuation  zone  that  are 
contaminated,  including  public  parks,  businesses,  schools,  and  homes. 
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With  the  continuation  of  the  heat  wave,  with  a  third  day  above  33°  C,  the  situation  is 
dire.  Air  quality  is  poor,  and  heat  exposure  is  also  becoming  an  issue.  Pregnant  women, 
the  elderly  and  children  are  all  at  high-risk  for  adverse  effects. 

Wednesday  11:00  am 

Wednesday  morning  the  city  is  reeling  from  the  upset  of  evacuation,  and  the  uncertainty 
surrounding  the  chemical  leak.  Carbonyl  Chloride  will  leave  residual  effects,  but  the 
degree  of  contamination  is  uncertain.  The  public  has  questions:  How  much  Carbonyl 
Chloride  entered  the  River,  the  city's  water  supply?  How  much  is  deposited  on  surfaces? 
Are  our  children  safe?  Have  we  already  been  contaminated?  When  can  we  go  home? 
Equally  there  are  elderly  and  those  with  mobility  restrictions  that  have  been  unable  to 
evacuate,  and  finding  and  helping  these  citizens  is  becoming  the  priority,  as  over  200 
people  have  already  fallen  seriously  ill,  and  there  appears  to  be  over  1 5  first  responders 
with  potentially  fatal  exposure. 

The  internet  chatter  on  all  social  networking  sites  contains  both  accurate  and  inaccurate 
infonnation.  Many  people  are  upset  at  the  dispersion,  with  many  family  members  stuck 
on  either  side  of  the  toxic  cloud.  Rumours  regarding  the  toxic  nature  of  the  tap  water 
stop  people  from  using  it  to  decontaminate  themselves  as  they  believe  this  increases  the 
risk  of  exposure  through  surface  contact. 

Wednesday  12:00  pm 

Parents  are  increasingly  concerned  about  the  latent  effects  of  the  phosgene  on  surfaces. 
Equally,  with  many  of  the  schools  closed,  there  are  significant  interruptions  for 
education  and  workplace  productivity.  It  has  been  reported  that  over  55  hospital  patients 
have  passed  away,  due  to  either  exposure  to  the  chemical  exposure,  or  through  the 
evacuation  process.  There  are  now  86  confirmed  fatalities  and  over  450  seriously  ill. 

Thursday  7:30  pm 

Many  of  the  first  responders  and  health  care  workers  are  becoming  ill  after  prolonged 
exposure  to  contaminated  patients.  Police,  fire  and  EMS  have  been  working  at  maximum 
capacity  for  three  days,  and  many  are  falling  ill  with  respiratory  difficulties. 

Friday  8:00  am 

Overnight  the  temperature  broke,  with  a  violent  thunderstonn,  high  winds  and  heavy 
rains.  After  four  days  apart  from  loved  ones,  people  are  getting  anxious  to  return  to 
“normal”.  Despite  the  continued  enforcement  of  the  evacuation  zone,  many  people  are 
surreptitiously  entering  their  homes  and  businesses,  to  collect  personal  belongings.  Some 
are  crossing  the  evacuation  zone  to  reach  loved  ones  on  the  other  side  of  the  chemical 
cloud.  A  number  of  people  have  been  arrested  in  order  to  maintain  the  evacuation  order; 
this  creates  a  serious  public  protest  about  heavy  handed  police  authority. 
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Friday  12:00  pm 

Authorities  are  quick  to  release  an  announcement  regarding  the  change  in  weather 
conditions  and  how  that  impacts  the  evacuation  zone.  People  are  asked  to  stay  out  for 
another  two  days.  Compliance  is  becoming  an  issue.  Internet  chatter  displays  a  public 
mistrust  in  the  authorities,  and  a  frustration  at  being  displaced  for  so  long.  Many  who 
have  successfully  entered  or  crossed  the  toxic  area  are  reporting  no  ill  effects,  which  is 
causing  more  and  more  people  to  attempt  to  enter  the  evacuation  zone. 

Task  #2 
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M.10  Scenario  Assessment  Guide:  Scenario  Assessment 
Questionnaire  -  Train  Derailment  Scenario 


This  form  is  designed  to  capture  a  reader’s  understanding  of  what  contributes  to  the  complexity  of  a  given 
scenario.  Potential  contributions  to  complexity  for  a  scenario  are  described  below  in  terms  of  impact, 
uncertainty  and  vulnerability,  with  specific  challenges  or  areas  noted  for  each.  After  reading  the  scenario, 
the  reviewer  is  asked  to  rate  the  degree  to  which  each  challenge  was  evident  in  the  scenario.  The  results 
will  be  compared  with  data  collected  from  other  reviewers  to  select  a  suitably  complex  scenario  for  use  in  a 
simulation  exercise. 

Q1  Overall,  which  of  the  following  descriptors  best  describes  the  scenario? 

□  Simple 

□  Complicated 

□  Complex 


Please  rate  the  extent  to  which  each  of  the  following  challenges  is  evident  in  the  scenario.  Use  the  scale 
ranging  from  0  (not  at  all  evident)  to  4  (strongly  evident). 


0 

i 

2 

3 

4 

Impact 

Q2 

Impact  severity 

(i.e.,  number  of  fatalities  /casualties, 
amount  of  physical  damage/  psychological 
trauma,  degree  of  social  disruption/multiple 
events) 

Q3 

Impact  scope 

(i.e.,  breadth  of  impact  on  people,  services, 
structures,  societal  functions ) 

Q4 

Impact  timing 

(i.e.,  no  threat/warning  phase,  sudden 
onset,  sustained  duration,  and/or  recurring 
event ) 

Q5 

Media  involvement 

(i.e.,  extensive  media  coverage,  lack  of 
access  to  official  sources,  inappropriate 
use  of  social  media  tools) 

Q6 

Political  processes 

(i.e.,  multiple  organizations  involved, 
multiple  jurisdictions,  and/or  multiple  levels 
of  government,  public  fear  and/or  mistrust 
in  officials) 

Uncertainty 
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0 

1 

2 

3 

4 

Q7 

Novelty  of  situation 

(i.e.,  novel  hazard,  malicious  cause,  lack  of 
experience,  lack  of  controllability,  and/or 
latent  impacts) 

Q8 

Anticipation  and  planning 

(i.e.,  lack  of  planning,  rehearsal,  and 
sharing  of  plans,  and/or  lack  of  integration 
of  plans) 

Q9 

Data  and  information 

(i.e.,  lack  of  access  to  information, 
inaccurate  information,  lack  of  credible 
information  sources,  excessive  volume  of 
data,  and/or  fast  speed  of  information  flow) 

Q10 

New  organizations  and  partners 

(i.e.,  presence  of  non-traditional 
organizations  and  partners,  and/or 
presence  of  emergent  organizations  or  ad- 
hoc  groups  that  seif-organize  to  fiii 
perceived  resource  gaps) 

Q11 

Changing  context 

(i.e.,  big  changes  in  situation  context,  fast 
speed  of  change,  actual/perceived 
negative  consequences  of  change,  and/or 
negative  impacts  of  intervention  (negative 
secondary  effects) 

Q12 

Flexibility  of  interpretive  frameworks 

(i.e.,  imposed/unilateral  processes,  lack  of 
common/shared  processes,  and/or  lack  of 
creativity/improvisation) 

Vulnerability 

Q13 

Economic  development 

(i.e.,  lack  of  resources,  lack  of  economic 
diversity,  unequal  distribution  of  wealth, 
and/or  economic  instability) 

Q14 

Social  capital 

(i.e.,  large  number  of  at-risk  populations, 
lack  of  social  support,  poor  citizen 
participation  in  volunteering  and  decision 
making,  poor  sense  of  community,  and/or 
excessive/insufficient  attachment  to  place) 

Q15 

Community  competence 

(i.e.,  political  instability,  poor  leadership 
skills,  low  literacy  and  education  levels, 
lack  of  experience  with  similar  hazards, 
and/or  lack  of  community  partnerships  and 
support) 
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0 

1 

2 

3 

4 

Q16 

Information  and  communication 

(i.e.,  unreliable  or  excessive  amounts  of 
extraneous  information,  public  opinion 
ignored,  lack  of  trusted  media 
spokesperson,  ineffective  risk  messaging, 
and/or  poor  communication  infrastructure 
capabilities) 

Q17 

Other  infrastructure 

(i.e.,  high  degree  of  interdependence 
among  critical  systems,  lack  of  redundant 
or  backup  systems,  and/or  weak  or  blocked 
distribution  networks ) 

Q18  In  your  estimation,  which  of  the  following  factors  contributed  most  to  the  level  of  complexity  in  the 
scenario?  Please  check  one  box  and  explain  your  choice. 

□  Impact 

□  Uncertainty 

□  Vulnerability 

□  All  of  the  above  (factors  influenced  the  assessment  equally) 

Please  explain  why: 


Please  rate  the  overall  level  of  complexity  of  the  scenario  on  a  scale  of  0  (not  at  all  complex)  to  4  (very 
complex). 


0 

1 

2 

3 

4 

Q19 

Overall,  1  would  describe  the  level  of 
complexity  for  the  scenario  as: 

Please  rate  your  level  of  agreement  with  the  following  statements  on  a  scale  of  0  (strongly  disagree)  to  4 
(strongly  agree).  _ _ _ _ 


0 

1 

2 

3 

4 

Q20 

The  first  part  of  the  scenario  is  realistic  / 
plausible. 

Q21 

The  second  part  of  the  scenario  is  realistic 
/  plausible. 

Q22 

The  timeline  for  the  scenario  is  realistic  / 
plausible. 
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Annex  N  Decision  Quality  Rating  Instrument 


Decision  Quality  Rating  Questionnaire 


Decision  Quality  Rating 

Please  rate  the  extent  to  which  the  decisions  reflect  the  following  items  on  a  scale  of  0  to  4  (0 
1  -  to  a  small  extent;  2  -  somewhat;  3  -  to  a  great  extent;  4  -  to  a  very  great  extent). 

-  to  a  very  small  extent; 

To  a  very 
small  extent 

0 

To  a  small 
extent 

1 

Somewhat 

2 

To  a  great 
extent 

3 

To  a  very 
great 
extent 

4 

Q1 

The  decisions  reflect  multiple 
perspectives  from  within  the  pod 

O 

o 

o 

o 

O 

Q2 

The  decisions  reflect  multiple 
perspectives  from  outside  the 
pod 

O 

o 

o 

o 

O 

Q3 

The  decisions  reflect 

innovative/creative 

characteristics 

O 

o 

o 

o 

O 

Q4 

The  decisions  reflect  a  unified, 
cohesive  approach 

O 

o 

o 

o 

O 

Q5 

The  decisions  reflect  shared 
information 

O 

o 

o 

o 

O 

Q6 

The  decisions  reflect  shared 

resources 

O 

o 

o 

o 

O 

Q7 

The  decisions  reflect  shared 
activities 

O 

o 

o 

o 

O 

Q8 

The  decisions  reflect  shared 
power/authority 

O 

o 

o 

o 

O 

Q9 

The  decisions  have  significant 
operational  gaps  or  challenges 

O 

o 

o 

o 

O 

Q10 

The  decisions  have  significant 
strategic  gaps/challenges 

o 

o 

o 

o 

O 
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Please  rate  your  level  of  agreement  with  the  following  statements  on  a  scale  of  0  to  4  (0  -  strongly  disagree;  1  - 
disagree;  2  -  neutral;  3  -  agree;  4  -  strongly  agree). _ 


Strongly 

Strongly 

disagree 

Disagree 

Neutral 

Agree 

agree 

0 

1 

2 

3 

4 

Q11 

The  pod  considered  the  impacts 
of  the  event  on  multiple  tiers  (first 
tier  is  the  direct  damages  to 
victims  and  infrastructure;  second 
tier  is  the  impact  on  core 
services;  and  third  tier  is  the 
impact  of  an  event  on  trust,  order 
of  civil  society,  and  social  fabric). 

O 

o 

o 

o 

O 

Q12 

The  pod  considered  the  impacts 
that  implementing  solutions 
would  have  on  multiple  tiers  (first 
tier  is  the  direct  damages  to 
victims  and  infrastructure;  second 
tier  is  the  impact  on  core 
services;  and  third  tier  is  the 
impact  of  an  event  on  trust,  order 
of  civil  society,  and  social  fabric). 

O 

o 

o 

o 

O 

Q13 

The  pod  considered  the 
psychosocial  implications  of  the 
event. 

O 

o 

o 

o 

O 

Q14 

The  pod  considered  the 
psychosocial  implications  that 
implementing  solutions  would 
have  on  the  population. 

O 

o 

o 

o 

O 

Q15 

The  pod  considered  general 
resource  constraints  and 
availability  in  the  decision  making 
process. 

O 

o 

o 

o 

O 

Q16 

The  pod  considered  whether  or 
not  specialized  resources  would 
be  required  to  implement 
solutions. 

O 

o 

o 

o 

O 

Q17 

The  pod  considered  the  time 
requirements  that  solutions  would 
require  to  be  implemented. 

o 

o 

o 

o 

O 

Q18 

Considering  the  time  and 
resources  available,  the  pod’s 
decisions  were  good  overall. 

o 

o 

o 

o 

O 
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Annex  O  PODS  Participant  Workbook  Materials 


Briefing  Information 

During  the  course  of  this  experiment  a  facilitator  will  be  on  hand.  The  facilitator  will  conduct  a 
debriefing  following  the  experiment.  Participation  in  this  experiment  is  voluntary. 

As  a  participant,  you  will  be  immersed  in  a  fictional  scenario,  asked  to  complete  two  tasks,  and 
asked  to  complete  a  series  of  questionnaires.  These  questionnaires  will  ask  questions  about  your 
background  in  the  field  of  emergency  response,  your  experience  working  on  the  two  tasks,  and 
on  how  you  interpreted  the  events  in  the  scenario. 

While  the  events  in  the  scenario  are  fictional,  certain  aspects  of  this  experiment  will  be  realistic. 
Tasks  will  have  time  limitations,  and  the  scenario  will  continue  whether  the  task  is  completed  or 
not.  Please  ensure  that  you  take  the  time  to  fill  out  all  questionnaires. 

Participant  Profile  Instructions 

Your  role  within  the  experiment  will  be  based  upon  your  experience  in  emergency  response. 
During  the  course  of  the  experiment,  please  consider  yourself  as  having  been  relocated  to 
Gapville,  where  your  job  resembles  your  current  occupation  with  respect  to  authority,  power, 
role  and  responsibilities.  You  will  be  performing  the  same  duties  that  you  are  usually  responsible 
for.  If  you  are  retired,  please  imagine  that  you  are  occupying  your  former  job,  in  the  city  of 
Gapville,  and  that  you  have  come  out  of  retirement  for  the  course  of  the  experiment. 

Gapville  Profile 

Located  in  the  Province  of  Anticipate,  in  the  Country  of  Reframe,  between  Ripple  River  and  the 
Lemyre  Mountain  Range  is  Gapville.  The  city  is  only  80  km  from  the  Provincial  Capital,  and  is 
only  20km  away  from  the  national  border  to  the  south.  Gapville  is  a  growing  city  with  a 
population  of  205,000  with  all  the  built  and  social  infrastructures  found  in  a  medium  sized  North 
American  city.  For  Example,  the  Gapville  Dam  and  Hydro  Electric  station  provides  power  to  the 
entire  town  and  the  GAPTransit  Commission  runs  a  Light  Rail  system  and  bus  routes. 

In  recent  years  Gapville's  Hi-Tech  sector  has  expanded,  creating  an  influx  of  immigration.  The 
leading  technology  company,  P-RAM  Communications  employs  over  one  quarter  of  the 
community.  Tourism  is  another  large  industry,  thanks  in  part  to  the  creation  of  the  P-RAM 
Pavillion  built  in  2004.  This  corporate-sponsored  center  draws  tourists  from  all  across  Anticipate 
and  the  Baseball  Stadium  is  home  to  the  Gapville  Hazards. 

Municipal  and  Provincial  services  are  located  in  the  Anticipate  Building,  beside  the  courthouse. 
The  Gapville  Police  Force  and  the  Fire  Department  each  have  two  divisions  across  the  city.  The 
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University  of  Gapville  is  a  fully  accredited  and  well  respected  institution,  as  is  the  McLaughlin 
Memorial  Hospital  which  houses  an  excellent  pre  and  post-natal  facility  -  Claire's  Hope. 

This  center  of  excellence  draws  patients  from  all  across  Reframe.  Gapville's  original 
entertainment  district  contains  the  renowned  Corneil  Museum  of  Natural  History  and  the  Old 
Town  Historical  Theatre.  Gapville  has  a  multi-faith  population  and  provides  several  churches  of 
different  denominations  along  with  a  mosque  and  Buddhist  temple.  Social  infrastructure 
includes  outreach  centers,  social  clubs,  and  neighbourhood  watch. 


164 


DRDC  CSS  CR  2011-32 


Gapville  in  Depth 

Please  use  this  table  for  reference  throughout  the  course  of  the  experiment  on  an  as  needed 
basis. 


Total  Population 

•  205,000  residents 

Geography 

•  In  the  province  of  Anticipate 

•  In  the  country  of  Reframe 

•  80  km  from  the  Provincial  Capital 

•  20km  away  from  the  national  border  to  the  south 

•  Located  in  a  valley,  600  feet  below  sea  level 

•  Ripple  River  is  located  to  the  North  West 

•  Lemyre  Mountain  range  is  located  to  the  North  East 

•  North  temperate  zone: 

o  Forest  fires  are  prevalent  in  the  autumn 
o  Floods  in  the  rainy  spring  season  are  an  annual  issue 
o  Ice  storms  in  the  winter  time  are  common  in  valley 

areas 

Utilities 

•  The  Ripple  River  Dam  and  Flydro  electric  station;  buried  power 
lines  throughout  the  city 

•  Natural  gas  piped  into  Gapville  from  major  urban  center, 
Primerton,  20  kilometres  away 

•  Ripple  River  provides  water  supply  directly  into  the  town 

Telecommunications 

•  GAPtv  Head  office 

•  Broadcast  tower  serves: 

o  The  local  media  outlet,  GAPtv 
o  The  Provincial  public  broadcast  service  AnticipateTV 
o  2  local  radio  stations 
o  Telephone 
o  Wireless  Internet 
o  Cellular  Telephone  Tower 

Transportation 

•  Freight  and  passenger  railway,  along  the  Reframe  National 

Rail  Line 

•  Gapville  Domestic  airport  serves  Reframe  National  and 
provincial  air  carriers 

•  The  Airport  also  provides  charter  flights  and  pilot  training 

•  Interprovincial  Expressway  serves  150,000  cars  and  trucks 
daily  from  Primerton  to  Response  with  2  on  and  off  ramps  in 
downtown  core 

•  GAPTransit  (GT)  Public  Transit  Main  Street  Terminal  Central 

Hub 

o  GT  light  rail  /  GT  bus  routes 
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Government 

•  Anticipate  Provincial  and  Reframe  Municipal  Building 

•  Gapville  Courthouse 

•  Post  office 

Security 

•  Gapville  Fire  Department:  2  divisions,  2  others  in  the  rural 

areas 

•  Gapville  Police  Department:  2  divisions  downtown 

•  Medium  Security  Prison 

Health 

•  McLaughlin  Memorial  Hospital: 

o  Claire's  Hope  Pre  and  Post  Natal  Care  Unit 

•  Center-town  Drop-In  Clinic 

Financial 

•  Reframe  National  Bank  Service  branch 

•  4  ATMs  throughout  town 

Social  Services 

•  Johnson  Estates  Community  Center: 

o  Child  care  facility 
o  Library 

o  Track  and  field  facility 
o  Swimming  pool 

•  South-End  food  bank  and  outreach  center 

Industry 

•  Forestry,  processing  and  distribution  of  raw  timber  across  the 
country  in  Industrial  Park 

•  Riding  Dairy  and  Vegetable  Farm,  shipping  locally  and 
provincially 

•  Johnson  Estates  golf  and  country  club  service 

•  Johnson  Estates  home  construction  industry 

•  Hi-tech  Sector,  largest  employer:  P-RAM  Communications 

Communities 

•  Johnson  Estates 

•  Desmarais  Heights 

•  Center-town 

•  Retirement  living  facility  also  runs  a  meal  delivery  service 

Tourism 

•  P-ram  pavilion,  corporate  sponsored  center  includes: 

o  Theatre,  including  standing  room  for  up  to  25,000 
o  Indoor  parking  for  2100 
o  Kid's  Village 
o  Horse  racing 

•  4  Airport  area  hotels  can  accommodate  over  3000  guests  each 
for  a  total  of  12,000  visitor  occupancy  at  any  time 

Retail  Market 

•  Ripple  River  Shopping  Mall 

•  Center-town  markets  &  cafes 

•  Asian  Market 

•  Little  Italy 
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Education 

•  2  Elementary  schools 

•  1  High  School 

•  University  of  Gapville 

Sports  and  Recreation 

•  Pinsent  Park 

•  Baseball  Stadium: 

o  Gapville  Hazards  Baseball  Team  draws  19,800  people 
on  game  night 

o  Underground  parking  for  1500 

•  Hiking  /  Skiing  in  the  Lemyre  Mountain  Range 

Arts  and 

Entertainment 

•  Old  Towne  Movie  Theatre 

•  Corneil  Museum  of  Natural  History 

•  Sociale  Nightclub 

•  Anticipate  Art  Gallery 

Religious  Services 

•  1  Catholic  Cathedral 

•  1  Protestant  Church 

•  1  Buddhist  Temple 

•  1  Muslim  Mosque 

Volunteer  Services 

•  Meals  with  Wheels 

•  Neighbourhood  Watch 

•  Helping  Hand  Immigration  Center  run  out  of  the  Mosque 
providing: 

o  Translation  services 
o  Employment  help 

Social  Clubs 

•  Johnson  Estates  Private  Country  Club 

•  Transportation  Worker's  Club 

•  Boy  and  Girls  Club  of  Reframe 

•  Italian  Club 

•  Gapville  Juniors  AAA  Baseball  League 

•  Tai  Chi  Society,  run  out  of  the  Temple,  and  using  Pinsent  Park 
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Situation  Report  #1 

Incident  Number:  RD001-1 1 

Incident:  Radiological  Bomb  at  the  SSMD  Warehouse 

Date:  Information  valid  as  of  Tuesday,  15th  October,  17:50  hours  (EST) 

Description  of  current  incident:  At  around  15:50,  Monday  October  14th,  a  package  exploded  in 
the  loading  bay  at  the  Sort  &  Send  Mail  Delivery  (SSMD)  warehouse.  The  explosion  was  from  a 
dirty  bomb,  originally  intended  for  the  GAPTV  television  station,  and  it  has  now  spread  several 
ounces  of  radioactive  Cesium  137  dust  in  the  southern  industrial  sector  of  the  city.  The 
contaminated  areas  include  the  South  End  LRT  station  and  the  International  Borders  and  Customs 
warehouse.  The  source  of  this  attack  and  the  motives  behind  it  are  unknown  at  this  time. 

Source(s)  of  reporting:  Original  source  of  reporting  from  responders  on  site,  and  Gap  TV 
Broadcasts. 

Current  actions  in  response:  Provincial  HazMat  teams  were  first  on  site  after  the  local 
responders,  and  are  conducting  investigations  at  this  moment,  with  the  help  of  municipal 
responders.  Public  Safety  and  Emergency  Preparedness  is  the  primary  federal  department  for 
this  incident,  including  public  communications  until  otherwise  advised. 

Federal  Police,  the  Security  Intelligence  Services  and  the  National  CBRN  team  are  clearing  the 
contamination,  tracking  down  all  the  potential  victims  of  radiation  exposure,  and  evacuating  areas 
determined  to  be  in  a  “hot  zone”.  The  Department  of  Defence,  the  Military  and  the  National 
Health  Agency  are  all  working  on-site  at  the  EOC. 

Assessment/Analysis: 

Seventeen  people  are  wounded,  and  the  six  closest  to  the  explosion  are  critically  ill.  Some  of  the 
victims  of  the  flying  glass  and  debris  show  signs  of  mild  to  moderate  radiation  exposure. 
Apparently  the  radiation  sickness  is  worse  if  the  victim  has  open  wounds.  There  is  also 
considerable  concern  around  pregnant  women  and  children,  as  they  are  the  most  seriously 
affected  by  the  long  term  effects  of  radiation. 

The  6  critically  ill  warehouse  workers  were  taken  to  the  McLaughlin  Memorial  Hospital  by 
ambulance. 

These  causalities  contaminated  the  Hospital’s  emergency  room,  and  some  of  the  patients  and 
health  care  workers  who  were  in  the  ER  at  the  time. 

The  Hospital  has  called  a  code  brown  for  hazardous  materials.  The  Hospital  also  has  activated  a 
Memorandum  of  Understanding  with  the  local  pulp  and  paper  mills  to  use  their  chemical 
protection  equipment,  and  self  contained  breathing  apparati. 

After  the  4pm  shift  change  at  the  Hospital,  some  contaminated  health  care  workers  went  out  in 
public  unknowingly  spreading  radiation  carried  on  their  skin  and  clothing.  The  contaminated 
ambulances  also  went  back  out  on  call  before  they  could  be  notified  of  their  radiation  exposure. 
Infantry  along  with  military  medics  are  assisting  the  overrun  at  McLaughlin  Memorial  Hospital 
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with  moving  critical  patients  to  auxiliary  treatment  centers  at  the  walk-in  clinic  and  the  Golden 
Acres  assisted  living  home. 


The  international  press  arrived  around  10pm  Monday  evening  in  Gapville  at  the  same  time  as 
representatives  from  NATO,  the  United  Nations  and  the  International  Atomic  Energy  Agency. 

Contamination  travelled  on  the  light  rail  train  and  went  through  the  downtown  core,  the  university 
campus,  Little  Italy,  some  parts  of  the  northern  industrial  park  and  the  airport. 

Provincial  Parks  and  Recreation  are  now  working  with  Police,  GAPTransit,  Airport  Authorities 
and  the  Transportation  Safety  Board  to  keep  traffic  flowing,  and  to  assist  in  the  shelter  and 
support  for  all  those  who  have  arrived,  and  for  all  those  who  have  been  displaced. 

Transportation  Safety  Board  has  agents  at  the  Gapville  Domestic  Airport.  They  report  elevated 
readings  at  the  airport  that  could  be  related  to  either  naturally  occurring  radon,  or  from  possibly 
more  malicious  sources.  Security  Intelligence  Services  have  arrived  at  the  airport  and  have  begun 
to  shut  down  the  terminal. 

Additional  notifications:  All  the  buildings  in  a  2  kilometre  radius  of  the  explosion  have  been 
evacuated.  This  includes  the  massive  international  Borders  and  Customs  warehouse,  as  well  as 
the  LRT  South  End  station,  and  two  high  rises;  one  is  an  apartment  building,  the  other  an  office 
tower. 

Due  to  the  issues  at  the  Customs  warehouse,  the  Border  Services  Agency  has  been  notified,  as 
well  as  the  Department  of  International  Affairs. 

The  Transportation  Safety  Board  and  Airport  Authority  are  working  with  the  Security 
Intelligence  Service  right  now  to  evacuate  the  terminal  and  are  redirecting  commercial  air  traffic 
to  the  provincial  capital  of  Primerton. 

Municipal  transit  services  have  also  been  halted  for  now,  until  we  can  track  down  all  the 
potentially  exposed  victims,  and  all  areas  of  the  city  that  have  unacceptable  levels  of  radiation. 

90  prisoners  at  the  Medium  Security  penitentiary  who  were  outside  in  the  exercise  yard  at  the 
time  of  the  explosion  may  have  been  exposed  to  radiation  as  well.  Correctional  Services  have 
been  notified. 

Issued  by:  Gapville  Unified  Coordination  Centre 

Email/courriel:  GOC  @Gapville.CA  Tel:  (100)  555-7000  Fax/Telecopieur:  (100)  555-0999 
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Public  Health  Agency  Press  Release  Inject 


/  /  k 

Public  Health  Agency 

GapVillg 


PRESS  RELEASE 

One  Thousand,  Three  Hundred  and  Twenty  (1320)  people  have  arrived  at  the  examination  centers 
set  up  in  the  hospital  parking  lot.  The  six  (6)  Sort  &  Send  warehouse  employees  who  were  nearest 
to  the  explosion  are  all  critically  ill.  The  paramedics  and  the  initial  ER  staff  who  treated  the 
warehouse  employees  were  also  exposed  to  moderate  levels  of  radiation  and  have  fallen  ill. 

The  seven  (7)  GapTransit  victims  are  in  stable  condition,  as  are  the  customs  warehouse  workers, 
and  the  unprotected  fire  fighters,  EMS  and  police  officers  originally  at  the  scene.  Their  symptoms 
may  not  appear  for  another  day  or  so,  but  their  long-term  cancer  risks  are  high. 

Eighty-two  (82)  other  people  have  been  hospitalized  and  quarantined  for  treatment  of  sub-lethal 
but  serious  effects  of  radiation  exposure;  another  one-hundred  and  sixty  five  (165)  are  following 
out-patient  treatments.  Many  of  the  workers  who  left  the  site  after  the  explosion  have  been 
tracked  down  or  have  arrived  at  the  hospital. 

Thirty  percent  (30%)  of  hospital  staff  is  working  overtime,  dealing  with  the  influx  of  patients, 
coupled  with  the  loss  of  employees  who  have  been  exposed  to  radiation.  There  have  been  seven 
(7)  confirmed  radiation  exposure  cases,  amongst  the  health  care  workers. 

Family  members  for  some  of  the  responders  and  workers  have  been  found  to  have  varying  levels 
of  exposure  as  a  result  of  the  contaminated  clothing  brought  home.  Potentially  exposed  children 
and  pregnant  women  are  now  the  focus  of  intensive  surveillance  and  monitoring. 

The  hospital  officially  request  support  from  the  military’s  engineers  and  medical  unit  to  set  up  an 
emergency  field  hospital.  Additionally,  the  hospital  requests  that  Military  Medics  take  care  of  the 
prisoners  who  were  exposed,  as  they  are  currently  at  the  end  of  their  resources  at  the  emergency 
treatment  center.  NGO’s  and  Volunteer  agencies  are  ready  to  offer  aid,  but  do  not  have  the 
appropriate  protective  gear  to  access  the  hot  zones.  More  resources  will  need  to  be  reallocated  to 
the  management  of  volunteers,  for  their  own  safety.  Memos  of  Understanding  (MOU’s)  have 
been  activated  with  the  Pulp  and  Paper  mills,  for  their  chemical  protective  gear.  These  are  being 
dispersed  to  the  responders.  The  Gapville  Red  Cross  have  sent  workers  in  droplet  protection 
suits  to  the  quarantine  centers,  to  donate  clothing  to  those  who  have  to  submit  theirs  for 
incineration,  and  to  usher  the  public  through  triage  at  the  field  hospital. 
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Situation  Report  #2 

Incident  Number:  RD001-1 1 

Incident:  Radiological  Bomb  at  the  SSMD  Warehouse 

Date:  Information  valid  as  of  Wednesday,  16th  October,  18:00  hours  (EST) 

Description  of  current  incident:  At  around  15:50  pm,  Monday  the  14th  of  October,  a  package 
exploded  in  the  loading  bay  at  the  Sort  &  Send  Mail  Delivery  (SSMD)  warehouse.  The  explosion 
was  from  a  dirty  bomb,  originally  intended  for  the  GAPTV  television  station,  and  it  spread 
several  ounces  of  radioactive  Cesium  137  dust  in  the  southern  industrial  sector  of  the  city. 

The  location  where  the  bomb  was  produced  has  been  identified  and  additional  areas  in  the  city 
have  been  exposed  to  contamination.  Evacuation  and  containment  activities  are  being  taken  by 
police  assisted  by  military  personnel.  Large  numbers  of  people  have  left  the  city  while  the 
movement  of  the  majority  of  the  population  has  been  restricted  to  ensure  public  safety  as  every 
attempt  is  being  made  to  determine  if  there  are  additional  exposures. 

Source(s)  of  reporting:  Original  source  of  reporting  from  Responders  on  Site,  and  GapTV 
Broadcasts. 

Current  actions  in  response:  The  federal  presence  and  the  military  arrived  following  the 
implementation  of  the  National  Emergency  Plan  following  a  request  by  the  province  and  the 
municipality.  Since  that  time,  additional  national  and  international  resources  have  been  arriving 
across  the  city. 

The  priority  continues  to  be  concentrating  on  public  safety  and  containment  of  sites.  Local  police 
with  the  military  assisting  them  in  providing  security  at  the  prison  on  the  exterior  as  well  as  at  the 
perimeter  of  various  contaminated  sites. 

The  military  will  continue  to  assist  the  hazmat  teams  and  the  forensic  investigators  in  doing  a  city 
wide  assessment  to  detect  any  additional  sites  and  if  discovered  they  will  provide  additional 
security  to  prevent  people  from  further  exposure.  The  current  taskings  for  all  national  groups  will 
be  centred  on  security  and  providing  humanitarian  assistance  under  the  direction  of  local 
authorities. 

Assessment/Analysis: 

SIS  agents  identified  those  responsible  for  the  attack  and  where  they  assembled  the  bomb.  The 
peipetrators  used  a  suburban  house  in  Gapville’s  upscale  neighbourhood,  the  Johnson’s  Estates. 
Radiation  levels  there  are  moderate  to  high. 

There  are  at  least  20  houses  that  will  need  to  be  decontaminated  or  destroyed.  The  Fire 
Department  is  working  with  HazMat  and  the  CBRN  Team  right  now,  evacuating  the  Johnson’s 
Estates.  They  are  conducting  a  door  to  door  canvas  in  the  affected  areas  to  ensure  residents  have 
evacuated. 

There  is  concern  that  the  perpetrators  possibly  contaminated  themselves  during  the  bomb 
assembly,  and  may  be  spreading  the  contamination  as  they  travel. 
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This  would  explain  the  unusually  high  radiation  readings  found  at  the  Gapville  airport. 

The  south-eastern  section  of  the  city  is  still  closed  off,  with  the  south  end  highway  exit  shut 
down.  Some  bus  routes  are  still  running,  but  the  Light  Rail  is  still  shut  down.  Police  are  asking 
that  citizens  do  not  cross  the  hot  zone  barriers 

The  municipal,  provincial  and  federal  teams  have  taken  a  variety  of  actions  intended  to  restore 
public  safety  and  confidence.  Overnight,  the  fire  stations  that  went  to  the  fire  have  been  closed 
and  their  equipment  and  vehicles  are  being  decontaminated.  The  ambulances,  the  hospital 
emergency  room  and  the  rooms  where  the  patients  were  handled  are  also  being  decontaminated. 

The  city  has  set  up  a  social  media  site,  with  Facebook  and  Twitter  feeds,  as  well  as  the  Ministry 
of  Resources  radiation  plume  maps. 

The  Red  Cross  and  other  volunteers  groups  have  set  up  temporary  schools  and  daycares  for  all  the 
children  who  cannot  attend  classes  today  at  the  Johnson  Public  School. 

There  is  a  concern  that  residents  and  business  owners  are  attempting  to  return  to  their  properties 
to  protect  them  as  rumours  and  media  reports  lead  them  to  believe  there  has  been  looting  in  the 
city.  This  is  placing  additional  strain  on  those  who  are  trying  to  maintain  the  security  and  safety 
perimeters. 

Additional  notifications: 

The  Department  of  Foreign  Affairs  is  also  working  with  investigators  over  the  international 
contamination  issues. 

Long  term  environmental  impact  assessments  are  being  requested. 

Issued  by:  Gapville  Unified  Command  Centre 

Email/courriel:  GOC  @Gapville.CA  Tel:  (100)  555-7000  Fax/Telecopieur:  (100)  555-0999 
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2  minutes  ago  Like  Comment 

Diane  Lemyre-Kenny  j  heard  that  Sante  Town  is  a  safe  place  to  go. 
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Debriefing 


Thank  you  for  taking  part  in  the  scenario.  Please  join  the  other  pod  members  for  a 
debriefing  session  in  the  plenary  room.  Feel  free  to  ask  the  facilitator  any  questions  that  you 
might  have.  Please  hand  in  your  binder  and  questionnaires  to  a  staff  member  in  the  plenary 
room. 

During  the  debriefing  session  you  will  be  asked  to  fill  out  an  additional  questionnaire  about 
how  you  interpreted  the  events  in  the  scenario. 

If  you  would  like  to  discuss  the  scenario  further,  please  contact  us  at  GAP-Sante. 

Louise  Lemyre 

Professor,  School  of  Psychology 
Faculty  of  Social  Sciences 
University  of  Ottawa 
Tel:  613-562-5800  (ext  1196) 

llemyre(g»uottawa.ca 

Gap-Sante 

Tel:  (613)  562-5800,  ext.  2321 
Fax:  (613)  562-5350 

gapsante(a»uottawa.ca 
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Annex  P 


Inter-GAP  In  Vivo  Session  Materials 


P.1  Briefing  Slide  Deck 


GAPSante 

— — - ► 

Welcome  to  the  PODS  Project 


Louise  Lemyre.  Ph.D.,  FRSC 

School  of  Psychology,  Faculty  of  Social  Sciences 
Director  of 'Groupe  d' Analyse  Psychosociale  de  la  sante1,  GAP-Sante 
McLaughlin  Research  Chair  on  Psychosocial  Aspects  of  Risk  and  Health 
Institute  of  Population  Health 
University  of  Ottawa 
louise  lemvre@uOttawa  ca 

vy>WLflapsanteupttayra_ca. 
www.  youtube  com/GAPSante 


Projet  PODS  Project 


©  Lemyre  etal  ,1010 


GAPSante 


Participant  Role 

•  View  fictional  scenario 

•  Complete  two  tasks 
Note:  Be  sure  to  fill  out  your 
own  worksheets  as  completely 
as  possible 

•  Fill  out  questionnaires 
after  each  task 

•  Debrief 


Projet  PODS  Project  ©  Lemyre , 


GAPSante 


Communication  Options 

•  Chat  function 

•  Two  way  radio 

Reminders. 

•  Set  personal  cell  phones  to 
"vibrate' 

■  Refrain  from  adjusting 
equipment 


Projet  PODS  Project  e  Lemyre  et«i,  20  to 


8 

- ► 

GAPVILLE 
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P.2  Controller  Script 

CONTROLLER  OPEN 


Time  &  Slide  Activity  Duties 


•  TECHNICAL: 

•  Plug  in  headset 

•  Start  Nefsis  Conference 

•  Ensure  all  sound  and  video  is  functioning 
properly 

•  REMEMBER* 

•  From  now  on,  control  mute  through  the 
headset,  not  the  software. 

•  MEDIA: 

•  Pre-load  all  media  in  THIS  order: 

•  PODS  Flow  2feb20 1 1 .  ppt 

•  PODS  Scenario  Part  l.wmv 

•  PODS  Scenario  Part  2.wmv 

•  PODS  Scenario  Open  Comml.wmv 

•  PODS  Scenario  Part  3.wmv 

•  PODS  Scenario  Youtube. wmv 

•  PODS  Scenario  Open  Comm2.wmv 

•  POWER  POINT: 

•  Begin  with  Flow  Slide  #1  Waiting  on 
Screen....  Ensure  the  Annotation  function  is 
deselected. 

•  LAYOUT: 

•  Select  Follow  my  Layout 

•  Select  Style  1 

TECHNICAL: 

Begin  Nefsis  Recording,  GO  TO:  top  left 
icon;  record  conference; 

Use  .avi  codec  and  save  to  desktop. 

REMEMBER* 

Record  Actual  Desktop.  Enable  audio  for 
participant  introduction. 

VOICE  OVER: 

“Good  Morning  PODS,  can  you  please 
introduce  yourselves  to  the  camera,  with 


09:00 


Participant  Identification 


In  turn,  beginning  with  POD  #1 , 

Participant  #1 ,  please  introduce  yourself 
to  your  web  camera,  and  the  video 
camera  in  the  room  -  using  your  full  name 
and  participant  number 


Thank  you 


Welcome 


07:30- 

09:00 


Setup 


Please  be  seated... 

The  PODS  Project 

will  begin  shortly.... 


A 
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your  name  and  participant  number,  starting 
with  you  POD  one. . 


Gapville 

Briefing 

Welcome  to  Gapville,  your  new 
hometown!  You  have  been  relocated  to 

• 

TECHNICAL: 

• 

Mute  Controller  Headset 

Gapville,  where  your  job  resembles  your 

current  occupation  with  respect  to 
authority,  power,  role  and  responsibilities. 

You  will  be  performing  the  same  duties 
for  which  you  are  currently  responsible. 

• 

GO  TO:  Audio;  Mute  All 

a 

rv^ii  roos  FKijaci  ©  lenryre  « II  2010 

• 

• 

POWER  POINT: 

Move  to  Flow  Slide  #3  -  leave  on  screen 

for  30  seconds. 

• 

VOICE  OVER: 

• 

Before  the  video:  “Feel  free  to  use  the 
color  map  of  Gapville  for  reference,  and 
you  have  a  black  and  white  copy  provided 
in  your  binder,  that  you  may  draw  and 
write  on  as  well.” 

• 

MEDIA: 

• 

GO  TO:  Layout;  Share  Only;  Full  Screen 

• 

Inside  the  MEDIA  PLAYER,  select  “Fit  in 
View”  and  turn  down  the  media  player 
volume  to  about  1/3 

• 

Play  the  Gapville  Intro 

09:10 

Envelope  #1 

Deliver 

Please  Open  Envelope  #1  which  contains  a 

Scenario 

POWER  POINT: 

Situation  Report  outlining  the  details  of  an 
extreme  event  in  Gapville. 

(1st  Half) 

• 

Advance  to  Flow  Slide  #4 

CURRENT  CONDITIONS  in  GAPVILLE 
.  5:30  pm  EST 

• 

Keep  on  screen  for  30  seconds. 

.  Tuesday  Oct.  15.  Mid-Autumn 

i  .  Cloudy.  Winds  at  3.2km/hr. 

.  2  degrees  Celsius 

C Lemyi* « u  2010 

• 

• 

• 

MEDIA: 

Play  VIDEO:  PODS  Scenario  Part  1 

Play  VIDEO:  PODS  Scenario  Part  2 

09:20- 

09:40 

Envelope  #2 

Begin  Task 

Please  Open  Envelope  #2,  and  perform 

Task  #1  with  the  worksheets  provided. 

1 

• 

• 

TECHNICAL: 

RIGHT  CLICK  on  all  the  POD  rooms  in 

You  have  20  minutes  to  complete  the  task 
within  your  pod 

the  user  list.  GO  TO:  Audio  Settings 

Use  the  Chat  Function  on  screen, 

• 

Adjust  POD  Speakers  to  0 

A.  or  the  CB  Radios  provided  to 

Kra  communicate  between  pods... 

• 

Ensure  POD  Microphone  is  still  set  at  90 

rmciWlNw  ©  lenryre  «  al  2010 

• 

• 

POWER  POINT: 

Advance  to  Flow  Slide  #5,  keep  on  screen. 

• 

• 

LAYOUT: 

GO  TO:  Share;  Chat;  Chat  with  all  users 

DRDC  CSS  CR  201 1-32 


179 


• 

GO  TO:  Layout;  Style  1;  Fullscreen 

• 

VOICE  OVER: 

• 

“I  want  to  remind  Participants  of  the  two- 
way  radios  and  highlight  the  chat  function 
on  the  right  side  of  the  screen. . .” 

• 

Enter  “Flello”  in  the  chat  box  as  an 
example. 

• 

Allow  Participants  20  Minutes  to  complete 
the  task.  Inform  participants  of  their  time 
remaining,  at  10  minutes,  and  5  minutes. 

09:40- 

Video  Conference  with  EOC 

Begin  EOC 

10:15 

We  are  now  opening  your  audio/video 
channels  Please  use  your  webcam  and 
computer  microphone  to  talk  “face-to-face" 
with  the  other  pods. 

Conference 

• 

POWER  POINT: 

• 

Advance  to  Flow  Slide  #6 

Use  the  next  35  minutes  to  discuss  your 

MEDIA: 

findings,  and  the  next  steps  forward.. 

• 

• 

Play  VIDEO:  PODS  Sit  Rep  Open  Comm  1 

• 

LAYOUT: 

• 

Save  Chat  file  to  Desktop  as  chat  date.rtf 

• 

GO  TO:  Layout;  Video  Only;  Fullscreen 

• 

TECHNICAL: 

• 

Unmute  All  Audio 

VOICE  OVER: 

• 

“Y ou  should  now  be  able  to  hear  each 
other.  You  have  35  minutes  to  complete  the 
task.” 

• 

Remind  participants  of  time  remaining,  at 

20,  10  and  5  minutes. 

10:15- 

10:30 

Task#1  Questionnaire 

Task  1 

Questionna 

• 

LAYOUT: 

Please  fill  out  the  task  questionnaires 
included  in  Envelope  #2. 

ire 

• 

Exit  Fullscreen  to  access  layout  panel. 

You  have  15  minutes  to  complete  the 
questionnaires... 

• 

GO  TO:  Layout;  Style  1;  Fullscreen 

4 

MatMtoosnotKi  _ ,  .  . 

©  lemyte  et  al  2010 

• 

• 

POWER  POINT: 

Advance  to  Flow  Slide  #7 

• 

VOICE  OVER: 

• 

“Once  you  have  finished  your 
questionnaires  we  will  have  a  short  break. 
The  experiment  will  resume  at  (10:50am)” 
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10:30- 

Break 

Break 

10:50 

Please  take  a  20  minute  break,  then  return 
to  your  pod  room. 

• 

• 

POWER  POINT: 

Advance  to  Flow  Slide  #8 

CLemvteet>|  M1)) 

10:50- 

Envelope  #3 

Deliver 

11:00 

.  Please  Open  Envelope  #3  which  contains 
a  Situation  Report  outlining  additional 
details  of  the  Radiological  event  in 

Gapville. 

CURRENT  CONDITIONS 

.  1 1 :30am  EST 

.  .  Wednesday  Oct.  16.  Mid-Autumn 

iA.  .  Cloudy,  Winds  at  1.4km/hr. 

.  5  degrees  Celsius 

©  Lemyre  et  al  2010 

Scenario 
(2nd  Half) 

• 

• 

• 

• 

POWER  POINT: 

Advance  to  Flow  Slide  #9  -  leave  on 
screen  for  30  seconds. 

MEDIA: 

Play  VIDEO:  PODS  Scenario  Part  3 

• 

Play  VIDEO:  PODS  Scenario  Part 

Y  outube 

11:00- 

Envelope  #4 

Begin  Task 

11:20 

Please  Open  Envelope  #4,  and  perform 

Task  #2  with  the  worksheets  provided. 

You  have  20  minutes  to  complete  the  task 
within  your  pod 

2 

• 

• 

• 

TECHNICAL: 

Adjust  all  POD  Speakers  to  0 

Ensure  POD  Microphone  set  at  90 

Use  the  Chat  Function  on  screen, 
or  the  CB  Radios  provided  to 
communicate  between  pods.  .. 

rwinmmlw  ©  Lemyre  « il  2010 

• 

• 

POWER  POINT: 

Advance  to  Flow  Slide  #10,  keep  on 
screen. 

• 

• 

• 

• 

LAYOUT: 

Exit  Fullscreen  to  access  control  panel. 

GO  TO:  Layout;  Style  1;  Fullscreen 

GO  TO:  Share;  Chat;  Chat  with  all  users 

• 

• 

VOICE  OVER: 

“Hello  PODS,  again  I’d  like  to  remind 

participants  of  the  two-way  radios  and  the 
chat  function  on  the  right  of  the  screen.” 

•  Allow  Participants  20  Minutes  to  complete 
the  task.  Inform  participants  of  their  time 
remaining,  at  10  minutes,  and  5  minutes. 
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11:20  -  Video  Conference  with  EOC 

11:55 


We  are  now  opening  your  audio/video 
channels  Please  use  your  webcam  and 
computer  microphone  to  talk  “face-to-face” 
with  the  other  pods. 

Use  the  next  35  minutes  to  discuss  your 
findings,  and  the  next  steps  forward.. 


4 


Begin  EOC 
Conference 


MEDIA: 

Play  VIDEO:  PODS  Sit  Rep  Open  Comm 
2 

LAYOUT: 

Exit  fullscreen  to  access  control  panel. 

Save  Chat  file  to  Desktop  as 

chat_task2_date.rtf 

End  Chat  Function 

GO  TO:  Layout;  Share  Only;  Fullscreen 

POWER  POINT: 

Advance  to  Flow  Slide  #11 

VOICE  OVER: 

“You  should  now  be  able  to  hear  each 
other.  You  have  35  minutes  to  complete  the 
task.” 

Remind  participants  of  time  remaining,  at 
20,  10  and  5  minutes. 

LAYOUT: 

Exit  fullscreen  to  access  control  panel. 

GO  TO:  Layout;  Video  Only;  Fullscreen 


11:55- 

12:10 


Task  #2  Questionnaire 


Please  fill  out  the  task  questionnaires 
included  in  Envelope  #4 
You  have  15  minutes  to  complete  the 
questionnaires... 


Task  2 

Questionna 

ire 


•  POWER  POINT: 

•  Advance  to  Flow  Slide  #12 

•  VOICE  OVER: 

•  “Once  you  have  finished  your 
questionnaires  we  will  have  a  debrief  and 
lunch  in  the  meeting  room,  (3120),  where 
we  had  our  initial  briefing.  Please  bring 
your  completed  worksheets,  questionnaires 
and  binder  back  to  the  room  with  you.” 


12:10- 

GAPSante 

Experiment 

12:15 

ends  •  POWER  POINT: 

Thank  You. 

•  Advance  to  Flow  Slide  #13 

•  TECHNICAL: 

4 

•  Stop  the  Nefsis  recording,  and  do 

W 

**..-005***  Clem*.*  .,.2010 

NOTHING  while  it  renders  and  saves  onto 
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the  desktop. 

•  Once  all  participants  have  left  their  pods 
GO  TO:  Icon  in  top  left,  and  End 
Conference. 
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CONTROLLER  CLOSED 

Time  &  Slide 


Duties 


Activity 


07:30-  a»r-*u-: _ 

Setup 

09:00 

• 

TECHNICAL: 

The  PODS  Project 

• 

Plug  in  headset 

will  begin  shortly.... 

• 

Start  Nefsis  Conference 

4 

• 

Ensure  all  sound  and  video  is  functioning 

properly 

• 

REMEMBER* 

• 

From  now  on,  control  mute  through  the 

headset,  not  the  software. 

• 

MEDIA: 

• 

Pre-load  all  media  in  THIS  order: 

• 

PODS  Flow  2feb201 1.  ppt 

• 

PODS  Scenario  Part  l.wmv 

• 

PODS  Scenario  Part  2.wmv 

• 

PODS  Scenario  Open  Comm  l.wmv 

• 

PODS  Scenario  Part  3.wmv 

• 

PODS  Scenario  Youtube. wmv 

• 

PODS  Scenario  Open  Comm2.wmv 

• 

POWER  POINT: 

• 

Begin  with  Flow  Slide  #1  Waiting  on 

Screen....  Ensure  the  Annotation  function  is 

deselected. 

• 

LAYOUT: 

• 

Select  Follow  my  Layout 

• 

Select  Style  1 

09:00 


Participant  Identification 


Welcome 


* 

I 

| 

p 

CL 

1 

c 

please 

• 

introduce  yoursed  to  the  video  camera  in 
the  room  -  using  your  full  name  and 

participant  number 

• 

Thank  you 

« 

• 

• 

POWER  POINT: 

Go  to  Flow  Slide  #2 

VOICE  OVER: 

Ask  Participants  to  Introduce  themselves  on 
camera,  with  Participant  ID  Number 


Gapville 


Welcome  to  Gapvile,  your  new 
hometown'  You  have  been  relocated  to 
Gapvv*e  where  your  Job  resembles  your 
current  occupation  with  respect  to 
aullwny,  power  role  and  responsibilities 
You  wall  be  performing  the  same  dirties 
tor  which  you  are  currently  responsible 

6  •? 


Briefing 


•  POWER  POINT: 

•  Move  to  Flow  Slide  #3  -  leave  on  screen  for 
30  seconds. 

•  VOICE  OVER: 
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•  Before  the  video:  “Feel  free  to  use  the  color 
map  of  Gapville  for  reference,  and  you  have 
a  black  and  white  copy  provided  in  your 
binder,  that  you  may  draw  and  write  on  as 
well.” 

•  MEDIA: 

•  Inside  the  MEDIA  PLAYER,  select  “Fit  in 
View”.  Turn  down  the  media  player  volume 
to  about  1/3 

•  Play  the  Gapville  Intro  video  AFTER 
allowing  30  seconds  of  screen  time  for  Slide 
#3 


09:10 


Deliver 


Please  Open  Envelope  81  which  contains  a 
Situation  Report  outlining  the  details  of  an 
extreme  event  in  Gapville 

CURRENT  CONDITIONS  in  GAPVILLE 
5  30  pm  EST 

Tuesday  Oct.  16.  M»d-Autumn 
Cloudy,  Winds  at  32km/br 
.  2  degrees  Cetoius 


Scenario 
(1st  Half) 


POWER  POINT: 

Advance  to  Flow  Slide  #4 

MEDIA: 

Play  VIDEO:  PODS  Scenario  Part  1 
Play  VIDEO:  PODS  Scenario  Part  2 


09:20- 

10:10 


Envelope  #2 


Begin  Task 
1 


•  POWER  POINT: 

•  Advance  to  Flow  Slide  #5,  keep  on  screen. 

•  VOICE  OVER: 

•  Allow  participants  50  minutes  to  complete 
the  task. 

•  Inform  participants  of  their  time  remaining, 
at  10  minutes,  and  5  minutes. 


10:10- 

10:25 


Task  #1  Questionnaire 


Please  fill  out  the  task  questionnaires 
included  in  Envelope  82 
You  have  IS  minutes  lo  complete  the 
questionnaires 


Task  1 

Questionna 

ire 


•  POWER  POINT: 

•  Advance  to  Flow  Slide  #6 


•  VOICE  OVER: 

•  “Once  you  have  finished  your  questionnaires 
we  will  have  a  short  break.  The  experiment 
will  resume  at  (10:50am)” 
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Break 


10:25- 

10:50 


10:50- 

11:00 


Break 


Please  take  a  20  minute  break  men  return 
to  your  pod  room 


Envelope  #3 


Please  Open  Envelope  #3  which  contains 
a  Situation  Report  outfirang  additional 
details  of  the  Radiological  event  in 
GapvtDe 

CURRENT  CONDITIONS 

1 1  30am  EST 

Wednesday  Oct  16,  Mid-Autumn 
k  Cloudy.  Wind*  at  1 4km/T>r 
9  5  degrees  Celsius 


Deliver 

Scenario 


(2nd  Half) 


•  POWER  POINT: 

•  Advance  to  Flow  Slide  #7 


TECHNICAL: 

Wait  for  all  participants  to  arrive  back 

POWER  POINT: 

Advance  to  Flow  Slide  #8 


11:00- 

11:50 


Envelope  #4 


Please  Open  Envelope  «  and  perform 
Task  #2  with  the  worksheets  provided 
You  have  SO  minutes  to  complete  the  task 
within  your  pod 


•  MEDIA: 

•  Play  VIDEO:  PODS  Scenario  Part  3 

•  Play  VIDEO:  PODS  Scenario  Part  Youtube 

Begin  Task 

2  •  POWER  POINT: 

•  Advance  to  Flow  Slide  #9,  keep  on  screen. 


«  .  ; 


VOICE  OVER: 

Allow  Participants  50  Minutes  to  complete 
the  task. 

Inform  participants  of  their  time  remaining, 
at  10  minutes,  and  5  minutes. 


11:50- 

12:10 


Task  #2  Questionnaire 

Task  2 

Please  fill  out  the  task  questionnaires 

Questionna 

• 

included  in  Envelope  #4 

You  have  15  minutes  to  complete  the 

ire 

• 

questionnaires 

> 

• 

• 

POWER  POINT: 

Advance  to  Flow  Slide  #10 

VOICE  OVER: 

“Once  you  have  finished  your  questionnaires 
we  will  have  a  debrief  and  lunch  in  the 
meeting  room,  (3120),  where  we  had  our 
initial  briefing.  Please  bring  your  completed 
worksheets,  questionnaires  and  binder  back 
to  the  room  with  you. 


12:10- 

OAPSantt 

Experiment 

12:15 

ends 

Thank  You. 

•  POWER  POINT: 

•  Advance  to  Flow  Slide  #11 

4 

•  TECHNICAL: 

•  Stop  the  Nefsis  recording,  and  do  NOTHING 
while  it  renders  and  saves  onto  the  desktop. 
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•  Once  all  participants  have  left  their  pods  GO 
TO:  Icon  in  top  left,  and  End  Conference. 
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P.3  Session  Administration  Information  Form 


Proj»t  POOS  Project 


Session  Administration  Information  Form 


Date  (dd/mm/yyyy): 


Session  Type  (check 
one): 


Pod  Number  One: 


Pod  Number  Two: 


Oj  University  of 
Ottawa  Students 

o2  Professional 
Students 

o3  Senior  Officials 

Pod  Status: 

o0Not  filled  with 
participants 

Oj  Connected 

o2  Closed 

o0Not  filled  with 
participants 

Oj  Connected 

o2  Closed 


Session  #: 

Task  Type  (check 
one): 


#  of  Participants  / 
Pod: 

o0None 
o:  One 
o2  Two 
o3  Three 

o0None 
o:  One 
o2  Two 
o3  Three 


GAPSante 


ox  Coordination 
o2  Collaboration 


Pod  Type: 

o0  Not  Applicable 
ox  Homogeneous 
o2  Mixed 

o0  Not  Applicable 
Oj  Homogeneous 
o2  Mixed 
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Pod  Number  Three: 

o0Not  filled  with 
participants 

Oj  Connected 

o2  Closed 

o0None 

Oj  One 

o2  Two 

o3  Three 

o0  Not  Applicable 

o-l  Flomogeneous 

o2  Mixed 

Pod  Status: 

#  of  Participants  / 
Pod: 

Pod  Type: 

Pod  Number  Four: 

o0Not  filled  with 
participants 

Oj  Connected 

o2  Closed 

o0None 

Oj  One 

o2  Two 

o3  Three 

o0  Not  Applicable 

o-l  Flomogeneous 

o2  Mixed 

Scenario: 

Location: 

Observer  (initials): 

Facilitator: 

Start  Time  /  End  Time 
(hh:mm): 

:  /  : 

Session  duration 
(hh:mm): 
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P.4  Technical  and  Process  Issues  Form 


GAPSante 

Qpupe  cMmlyse  PyhoaxfateuOmw 


Date 

(dd/mm/yyyy): 

Session  #: 

Audio 

Simulation 

Delivery 

Problem(s): 

Oj  Yes 

o2  No 

Nature  of  the  problem: 

Video 

Simulation 

Delivery 

Problems(s): 

Oj  Yes 

o2  No 

Nature  of  the  problem: 

Participant 

Audio 

Recording 

Problem(s): 

Oj  Yes 

o2  No 

Nature  of  the  problem: 

Participant 

Video 

Recording 

Problems(s): 

ox  Yes 

o2  No 

Nature  of  the  problem: 

Inter-pod 
Communication 
Problem(s)  due 
to  Technology: 

Oj  Yes 

o2  No 

Nature  of  the  problem: 

4 

p.o/.tPODSp,o(«t  Technical  and  Process  Issues  Form 
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Conferencing 

Software 

Problem(s): 

Oj  Yes 

o2  No 

Nature  of  the  problem: 

Internet 

Connection 

Failure(s): 

ox  Yes 

o2  No 

Nature  of  the  problem: 

Other  Technical 
Issue(s): 

Oi  Yes 

o2  No 

Nature  of  the  problem: 

Participant(s) 

Absent: 

ox  Yes 

o2  No 

Nature  of  the  problem: 

Participant(s) 

Late: 

Oj  Yes 

o2  No 

Nature  of  the  problem: 

Participant(s) 
Left  Early: 

Oj  Yes 

o2  No 

Nature  of  the  problem: 

Consent 

Refusal(s): 

o:  Yes 

o2  No 

Nature  of  the  problem: 

Delayed  Start 
Time: 

Oi  Yes 

o2  No 

Nature  of  the  problem: 

Session 

Interruption(s): 

ox  Yes 

o2  No 

Nature  of  the  problem: 
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Participant(s)  o1Yes 
Confused  by 

Directions:  °2  No 

Nature  of  the  problem: 

Participant(s)  o1Yes 
Confused  by 

Software:  °2  No 

Nature  of  the  problem: 

Other  Process  OiYes 
Issue(s): 

o2  No 

Nature  of  the  problem: 
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Annex  Q  Recruitment  Documents 


Q.1  Student  Recruitment  Pool  Posting 


Study 

Information 


studyName  Understanding  Problem  Solving  and  Decision 
Making 

Description  Please  arrive  half  an  hour  early  (arrive  at  8:30am)to  be  briefed  on  study  procedures  and 

experiment  schedule.  The  session  will  take  place  in  English  and  will  last  approximately  three  hours 
and  will  resemble  a  table-top  exercise  often  used  for  emergency  planning.  The  session  will  take 
place  at  the  University  of  Ottawa  from  9am-12pm.  As  a  participant  you  will  be  asked  to  interact 
with  other  session  participants  as  they  work  through  the  exercises.  As  well,  you  will  be  asked  to 
complete  various  questionnaires  on  perceptions  and  opinions  of  tasks,  performance  and 
interactions.  All  sessions  will  be  video  and  audio  recorded  for  data  collection  purposes.  All  aspects 
of  participation  are  voluntary.  You  will  be  asked  to  read  through  and  sign  a  consent  form  indicating 
informed  consent  prior  to  participation  in  the  session.  **Pizza  lunch  will  be  included  at  12:30pm 


Prescreen 

Restrictions 


No  Restrictions  -  [View/Modify  Restrictions] 


Duration  180  minutes 

Points  2  Points 

Researcher  Hilary  Kitchener 

Email:  hilary.kitchener@gmail.com 

Participant  24  hours  before  the  study  is  to  occur 
Sign-Up 
Deadline 

Study  Status  Visible  to  participants  (approved) 

Active  study  (appears  on  list  of  available  studies) 

REB/Comite  08-10-31  (expires  September  13,  2011) 

d'ethique 

Approval 

Code 
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Q.2  Participation  Invitation  Letter 

Dear  <insert  name>, 

I  am  inviting  you  to  participate  in  a  study  at  the  University  of  Ottawa.  Our  research  team  has 
received  your  name  from  <insert  references  Please  see  below  for  a  brief  description  of  our 
study.  We  believe  that  participants  will  find  it  an  interesting  experience. 

Problem-solving  and  Organizational  Decision-making  Simulation  (PODS)  Project 

Needed:  Participants  for  an  innovative  simulation  exercise  on  problem  solving  and  decision 
making  within  the  context  of  an  extreme  event.  The  research  is  being  conducted  by  the  University 
of  Ottawa  and  funded  by  Defence  Research  and  Development  Canada  (DRDC). 

Purpose  of  the  study:  To  examine  how  officials  work  together  during  emergencies  which  reguire 
joint  responses  from  both  traditional  first  responders  (police,  fire  and  EMS,  public  health)  and 
non-traditional  ones  (social  and  community  services,  non-governmental  organizations  or 
volunteer  groups)  as  well  as  from  the  military  or  other  government  agencies.  The  overall  objective 
of  the  research  is  to  identify  ways  to  improve  interoperability  between  organizations  during  a  large 
scale  emergency  event  or  threat. 

Who:  Senior  level  emergency  management  practitioners  with  at  least  one  year  of  experience  with 
high-level  decision-making  in  managing  and  responding  to  emergencies. 

When:  Session  dates  include: 

February  23rd  -  Wednesday 
February  24th  -  Thursday 
February  25th  -  Friday 
February  28th  -  Monday 
March  1st-  Tuesday 
March  2nd  -  Wednesday 
March  3rd  -  Thursday 
March  23rd  -  Wednesday 

Sessions  will  be  held  in  the  mornings  between  8:30-12:30.  A  light  lunch  will  be  provided. 

Where:  Desmarais  Hall,  Room  3120,  University  of  Ottawa  (55  Laurier  East,  KIN  6N5) 

What:  During  sessions,  participants  will  be  assigned  to  groups  of  three  participants,  called  a 
"pod".  These  pods  will  be  immersed  in  a  table-top  exercise,  which  is  driven  by  a  CBRNe  scenario 
and  situated  in  the  fictional  mid-sized  city  of  “GAPVille”.  The  participants  will  work  in  small  groups 
on  a  series  of  tasks  to  address  particular  aspects  of  the  emergency.  In  some  cases,  participants 
will  be  communicating  with  the  other  pods  with  the  aid  of  video-conferencing  software.  All  aspects 
of  participation  are  voluntary.  You  will  be  asked  to  read  through  and  sign  a  consent  form 
indicating  informed  consent  prior  to  participation  in  the  session. 

Please  contact  Hilary  Kitchener  at  qapsante@uottawa.ca  or  (613)  562-5800  x2321  to  indicate 
your  interest  in  participating.  We  will  contact  you  to  confirm  arrangements. 

Sincerely, 

<insert  name  &  signature> 
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Q.3  Phone  Confirmation  Script 


Phone  Confirmation  Script 


“Hello, 

My  name  is  [first  name  of  caller],  I  am  calling  from  GAP-Sante  regarding  your  participation  in 
our  Problem-solving  and  Organizational  Decision-making  Simulation  (PODS)  Project. 

In  order  to  meet  the  criteria  for  participation  I  need  to  ask  you  five  questions. 

1 .  Have  you  had  previous  experience  in  both  strategic  and  operational  decision  making  during 
at  least  one  major  event? 

2.  Have  you  been  in  a  management  level,  decision  making  role  at  your  organization  for  at  least 
12  months? 

3.  Do  you  have  the  authority  to  make  decisions  regarding  the  allocation  of  resources  on  behalf 
of  your  organization? 

4.  Are  you  able  and  willing  to  participate  in  this  study? 

5.  Are  you  able  and  willing  to  participate  in  English?” 

[If  ANY  answer  is  NO:] 

“We  really  appreciate  you  taking  the  time  to  answer  these  questions.  Unfortunately  we  will  be 
unable  to  ask  you  to  participate  in  this  study,  at  this  time. 

Once  again,  thank  you  for  your  time.” 

[If  ALL  answers  are  YES:] 

“Great.  I  would  like  to  confirm  your  participation,  and  select  a  date  for  your  session.  Which  of 
these  dates  would  suit  you  best?  We  have  openings  on  . . .” 

[Refer  to  dates  with  open  time  slots  and  schedule  a  mutually  agreeable  timeslot  for  the 
participant] 

“The  study  is  being  conducted  on  campus  at  the  University  of  Ottawa  in  the  Demarais  building. 
The  address  of  the  building  is  55  Laurier  East.  Please  meet  us  on  the  third  floor  in  room  3 120  at 
8:30  am. 

A  few  days  before  your  session,  I  will  e-mail  you  to  remind  you  of  the  appointment.  If  you  have 
to  cancel,  please  contact  us  at  gapsante@uottawa.ca  or  (613)  562-5800  Ext.  2321. 

I  look  forward  to  meeting  you  on  [date  of  session].  Thank  you  very  much  for  helping  us  with  our 
research.” 
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Q.4  Reminder  E-mail 


Dear  <insert  participant  name>, 

This  is  a  reminder  of  your  upcoming  appointment  to  participate  in  the  Problem-solving  and 
Organizational  Decision-making  Simulation  (PODS)  Project  experiment  on  <insert  date>. 

Please  come  to  room  3120  on  the  3rd  floor  of  the  Desmarais  Building  located  at  the  University  of 
Ottawa;  there  will  be  signs  to  help  direct  you.  The  session  begins  at  8:30  am  and  runs  until  12:30 
pm.  A  light  lunch  will  be  served  following  the  session. 

The  Desmarais  Building  is  located  at  55  Laurier  Avenue  East  (click  here  for  a  link  to  our  location  on 
Google  Maps).  If  you  arriving  by  bus,  please  exit  at  Laurier  Station.  Directions  to  the  Desmarais 
Building  by  car  can  be  found  following  this  e-mail. 

Underground  parking  is  available  in  the  Desmarais  Building  (at  a  cost  of  $4.00  an  hour,  prepaid). 
In  order  to  access  the  parking,  you  must  take  the  417E  exit  at  Nicholas  Street.  Next,  take  your 
first  right  after  you  go  across  the  Laurier  intersection.  Alternate  options  for  parking  are  available 
in  the  Byward  Market  or  at  the  Rideau  Centre. 

If  you  have  any  questions  about  the  study,  please  do  not  hesitate  to  contact  Hilary  Kitchener  by  e- 
mail  at  hkitc036@uottawa.ca  or  by  phone  at  613-562-5800  ext.  2321.  If,  for  any  reason,  you  need 
to  cancel  your  appointment  please  let  us  know  as  soon  as  possible. 

Sincerely, 

cinsert  name  of  sender> 


Directions  to  Desmarais  Building,  University  of  Ottawa 

Desmarais  Building  from  the  East 

The  main  access  to  Ottawa  from  the  East  is  via  Highways  417  and  174. 

•  Off  Highway  417  West,  take  the  Nicholas/Mann  exit 

•  Follow  the  Nicholas  Street  exit  to  Laurier  Avenue.  (Continue  straight  ahead,  do  not  turn 
on  Laurier) 

•  Turn  right  (east)  immediately  after  Laurier  Avenue  intersection  to  access  the  Desmarais 
Building  parking  lot. 

Desmarais  Building  from  the  West 

The  main  access  to  Ottawa  from  the  West  is  via  Highways  417  and  7. 

•  Off  Highway  417  East,  take  the  Nicholas/Lees  exit. 
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•  Follow  Nicholas  Street  to  the  first  set  of  lights  (Laurier  Avenue)  (Continue  straight 
ahead,  do  not  turn  on  Laurier) 

•  Turn  right  (east)  immediately  after  Laurier  Avenue  intersection  to  access  the  Desmarais 
Building  parking  lot. 

Desmarais  Building  from  the  South 

The  main  access  to  Ottawa  from  the  South  is  via  Highways  16,  3 1  or  416. 

From  Highway  16: 

•  Follow  Prince  of  Wales  Drive  to  Carling  Avenue 

•  Turn  right  (east)  on  Carling  Avenue 

•  Follow  Carling  Avenue  to  Bronson  Avenue 

•  Turn  left  (north)  on  Bronson  Avenue 

•  Follow  Bronson  Avenue  to  Laurier  Avenue 

•  Turn  right  (east)  on  Laurier  Avenue 

•  Follow  Laurier  Avenue  to  Waller  Street 

•  Turn  right  (south)  on  Waller  Street,  and  follow  loop  around  the  condominiums  to 
Nicholas  Street 

•  Turn  right  (north)  on  Nicholas  Street 

•  Turn  right  (east)  immediately  after  Laurier  Avenue  intersection  to  access  the  Desmarais 
Building  parking  lot. 

From  Highway  3 1 : 

•  Follow  Bank  Street  to  Isabella  Street 

•  Turn  right  (east)  on  Isabella  Street 

•  Off  Isabella  Street,  take  the  Highway  417  East  on-ramp 

•  Off  Highway  417  East,  take  the  Nicholas/Lees  exit. 

•  Follow  Nicholas  Street  to  the  first  set  of  lights  (Laurier  Avenue)  (Continue  straight 
ahead,  do  not  turn  on  Laurier) 

•  Turn  right  (east)  immediately  after  Laurier  Avenue  intersection  to  access  the  Desmarais 
Building  parking  lot. 

From  Highway  416: 

•  Take  the  Highway  417  East  on-ramp 

•  Off  Highway  417  East,  take  the  Nicholas/Lees  exit. 

•  Follow  Nicholas  Street  to  the  first  set  of  lights  (Laurier  Avenue)  (Continue  straight 
ahead,  do  not  turn  on  Laurier) 

•  Turn  right  (east)  immediately  after  Laurier  Avenue  intersection  to  access  the  Desmarais 
Building  parking  lot. 

Desmarais  Building  from  the  North  (Quebec) 

The  main  access  to  Ottawa  from  the  North  is  via  Highways  5,  50  and  148  in  Quebec. 
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•  Follow  the  indications  for  Ottawa  via  Macdonald  Cartier  Bridge 

•  Off  the  Macdonald  Cartier  Bridge,  follow  the  indications  for  King  Edward  Avenue 

•  Follow  King  Edward  Avenue  to  Laurier  Avenue 

•  Turn  right  (west)  on  Laurier  Avenue 

•  Follow  Laurier  Avenue  to  the  third  set  of  lights  (Nicholas  Street) 

•  Turn  right  (north)  on  Nicholas  Street 

•  Turn  right  (east)  to  access  the  Desmarais  Building  parking  lot. 
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Q.5  Thank  You  Letter  (pilot  testing) 


Universite  d’Ottawa  •  University  of  Ottawa 


<Insert  date> 


Dear  <Ms./Mr.  insert  last  name>, 

On  behalf  of  Gap-Sante,  we  would  like  to  take  this  opportunity  to  thank  you  for  your  invaluable 
input  into  the  Problem-solving  and  Organizational  Decision-making  Simulation  (PODS)  Project 
at  the  University  of  Ottawa,  funded  by  Defence  Research  and  Development  Canada  (DRDC),  the 
Natural  Sciences  and  Engineering  Research  Council  of  Canada  (NSERC),  and  the  Social  Sciences 
and  Humanities  Research  Council  (SSHRC).  With  the  help  of  participants  such  as  yourself,  this 
project  will  contribute  to  a  better  understanding  of  inter-organizational  relationships  and  shared 
decision-making  in  the  context  of  emergency  events.  . 

Again,  with  my  colleagues  Wayne  Comeil,  Paul  Boutette,  Celine  Pinsent,  and  our  whole  team  of 
graduate  students  and  research  assistants,  we  sincerely  appreciate  your  contribution  of  time  as 
well  as  your  thoughtful  participation  in  the  PODS  project.  We  acknowledge  your  support  for  the 
advancement  of  knowledge  in  emergency  response  and  psychosocial  preparedness  for  extreme 
events  in  Canada. 


Sincerely, 


Louise  Lemyre,  Ph.D,  MSRC  /  FRSC 

Ecole  de  psychologie  /  School  of  Psychology 

Faculte  des  sciences  sociales  /  Faculty  of  Social  Sciences 

Institut  de  sante  des  populations  /  Institute  of  Population  Health 

Universite  d'Ottawa  /  University  of  Ottawa 

55  Laurier  E  (3215)  Pavilion  Desmarais  Hall 

Ottawa,  ON,  CANADA  KIN  6N5 

tel:  +1  613-562-5800  (xll96) 

fax:  +1  613-562-5350 
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Q.6  Thank  You  Letter  (senior  level  participants) 


Universite  d’Ottawa  •  University  of  Ottawa 


<Insert  date> 


Dear  <Ms./Mr.  insert  last  name>, 

On  behalf  of  Gap-Sante,  we  would  like  to  take  this  opportunity  to  thank  you  for  your  invaluable 
input  into  the  Problem-solving  and  Organizational  Decision-making  Simulation  (PODS)  Project 
at  the  University  of  Ottawa.  With  the  help  of  participants  such  as  yourself,  this  project  will 
contribute  to  a  better  understanding  of  inter-organizational  relationships  and  shared  decision¬ 
making  in  the  context  of  emergency  events. 

This  project  is  funded  by  Defence  Research  and  Development  Canada  (DRDC),  the  Natural 
Sciences  and  Engineering  Research  Council  of  Canada  (NSERC),  and  the  Social  Sciences  and 
Humanities  Research  Council  (SSHRC).  The  overall  objective  of  the  research  project  is  to 
identify  ways  to  improve  collaboration  between  the  Canadian  Forces,  emergency  response 
professionals,  and  other  non-traditional  partners  during  extreme  events.  To  this  end,  as  a  senior 
level  actor  with  considerable  experience  pertaining  to  these  types  of  events,  your  participation  has 
been  indispensable  to  the  PODS  in  vivo  experiment.  Results  from  the  experiment  will  be  used  to 
formulate  recommendations  for  effective  communication  and  shared  decision  making  within  the 
context  of  the  Incident  Command  Structure. 

Again,  with  my  colleagues  Dr  Wayne  Comeil,  Dr  Celine  Pinsent,  Paul  Boutette,  and  our  whole 
team  of  doctoral  students  and  research  assistants,  we  sincerely  appreciate  your  contribution  of 
time  as  well  as  your  thoughtful  participation  in  the  PODS  project.  We  acknowledge  your  support 
for  the  advancement  of  knowledge  in  emergency  response  and  psychosocial  preparedness  for 
extreme  events  in  Canada. 


Sincerely, 


Louise  Lemyre,  Ph.D,  MSRC  /  FRSC 
Ecole  de  psycho logie  /  School  of  Psychology 
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Faculte  des  sciences  sociales  /  Faculty  of  Social  Sciences 
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Universite  d'Ottawa  /  University  of  Ottawa 
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Annex  R  Full  Pod  Session  Equipment 
Requirements 


Quantity 

Item 

Description 

Recommendation 

3 

POD  rooms 

Rooms  sufficiently  large  enough  to  comfortably 
hold  three  participants  as  well  as  a  computer  and  all 
camera  equipment.  Each  room  will  require  a  wired 
internet  connection. 

Rooms  should  be  in  close 
proximity  to  each  other  to 
assist  in  troubleshooting 

1 

Control  room 

Room  large  enough  for  at  least  two  administrators 
and  two  computers.  Ideally,  it  could  also  have 
enough  space  for  observers.  Like  the  POD  rooms,  it 
must  also  have  a  wired  internet  connection. 

3 

Web  camera 

Camera  software  may  need  to  be  installed  onto  the 
computers  prior  to  starting  the  experiment. 

Logitech  Web  cam  C260 

3 

USB 

Conferencing 

mics/speakers 

Web  cameras  often  are  equipped  with  internal 
microphones,  but  they  create  feedback.  USB 
conferencing  combined  microphone/speaker  sets 
solve  this  issue. 

Phoenix  PCS  duet 
conference  phone 

10 

Computers 

Four  computers  for  open  and  closed  Pods.  These 
computers  should  include  a  large  monitor  suitable 
for  sharing  among  several  people.  Computers 
should  also  have  a  minimum  of  three  USB  slots  as 
they  will  be  used  for  the  microphones,  web  cameras 
and  printers.  Computers  must  also  be  running 
Windows  XP  or  higher 

Two  computers  for  the  control  room.  One  the  host 
the  open  Pods  and  one  to  host  the  closed  Pods. 

Four  computers  for  observers  to  watch  the  session 
either  locally  or  remotely. 

Any  brand  of  PC 

3 

Video  camera 

Because  the  PODS  experiment  lasts  several  hours 
cameras  should  have  internal  hard  drives  suitable  for 
recording  up  to  three  hours  consecutively. 

Sony  DCRSX63  or  higher 

3 

Magic  Arms 

These  products  can  be  purchased  through  most 
photography  stores  and  will  be  used  to  install  the 
video  cameras  in  each  room.  Important:  please 
ensure  that  camera  mounts  are  included  with  each 
magic  arm. 

Manfrotto  143  Magic  Arm 
Complete 

2 

Microphone/ earp 
hone  headset 

These  will  be  used  in  the  control  room. 

Logitech  USB  Headset 
H360 

4 

Walkie  talkies 

These  will  be  used  as  an  alternative  communication 
method  between  PODS 

Cobra  CXT85C  GMRS 
radios 
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Quantity 

Item 

Description 

Recommendation 

4 

Digital  Voice 
recorder 

One  recorder  will  be  set  up  to  record  Walkie  Talkie 
communications. 

The  remaining  three  will  be  set  up  in  each  Pod  to 
record  all  Pod  audio. 

We  opted  to  use  an  extra 
laptop  and  recorded 
through  its  internal 
microphone. 

16 

Licences  internet 
web 

conferencing 

This  will  be  the  method  for  deploying  information  to 
the  pods  participants.  It  will  also  serve  as  a  platform 
for  communication  between  PODS.  Participants  are 
free  to  speak,  chat  or  even  email  between  PODS 

NEFS1S  web  conferencing 
NEFSIS.COM 
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List  of  symbols/abbreviations/acronyms/initialisms 


ANCOVA 

Analysis  of  Covariance 

ANOVA 

Analysis  of  Variance 

ATM 

Automated  Teller  Machine 

BBQ 

Barbeque 

C2 

Command  and  Control 

CB 

Citizens’  Band 

CBRN 

Chemical,  Biological,  Radiological,  Nuclear 

CBRNe 

Chemical,  Biological,  Radiological,  Nuclear,  Explosive 

CF 

Canadian  Forces 

Co. 

Company 

CORA 

Centre  for  Operational  Research  and  Analysis 

CRTI 

Chemical,  Biological,  Radiological,  Nuclear  and  Explosives  Research  and 
Technology  Initiative 

CSS 

Centre  for  Security  Sciences 

Demo 

Demonstration 

DMS 

Demarais 

DND 

Department  of  National  Defence 

DRDC 

Defence  Research  &  Development  Canada 

DRDKIM 

Director  Research  and  Development  Knowledge  and  Information 
Management 

EMO 

Emergency  Measures  Organization 

EMS 

Emergency  Medical  Services 

EOC 

Emergency  Operations  Centre 

ER 

Emergency  Room 

FC 

Forces  canadiennes 

Feds 

Federal  Government 

G8 

Group  of  Eight 

G-20 

Group  of  Twenty 

GAP 

Groupe  d’Analyse  Psychosociale 

I  lazMat 

Hazardous  Materials 

HC1 

Hydrogen  Chloride 
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ICS 

Incident  Command  System 

ISPR 

Psychology  Integrated  System  of  Participation  in  Research 

GT 

Gap  Transit 

ICU 

Intensive  Care  Unit 

ID 

Identification 

MANCOVA 

Multivariate  Analysis  of  Covariance 

MANOVA 

Multivariate  Analysis  of  Variance 

MOU 

Memorandum  of  Understanding 

MP3 

MPEG- 1  Audio  Layer  3 

NGO 

Non-governmental  Organization 

NI 

Non-Incident  Command  System 

NIMS 

National  Incident  Management  System 

No. 

Number 

Non-ICS 

Non-Incident  Command  System 

NSERC 

Natural  Sciences  and  Engineering  Research  Council  of  Canada 

PC 

Personal  Computer 

PODS 

Problem-solving  and  Organizational  Decision-making  Simulation 

PRiMer 

Psychosocial  Risk  Manager 

R&D 

Research  &  Development 

RCMP 

Royal  Canadian  Mounted  Police 

SARS 

Severe  Acute  Respiratory  Syndrome 

SDM 

Shared  Decision  Making 

SPSS 

Statistical  Package  for  the  Social  Sciences 

SSHRC 

Social  Sciences  and  Humanities  Research  Council 

SSMD 

Sort  &  Send  Mail  Delivery 

TA 

Technical  Authority 

TBD 

To  be  decided 

TIF 

Technology  Innovation  Fund 

TCPS 

Tri-Council  Policy  Statement 

uOttawa 

University  of  Ottawa 

USB 

Universal  Serial  Bus 
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